Steganographic camera communication

ABSTRACT

Identifiers or references to supplemental information or content regarding images may be steganographically encoded into the images. The identifiers or references may be encoded into least significant bits or less significant bits of pixels within the image that may be selected on any basis. The identifiers or references may include alphanumeric characters, bar codes, symbols or other features. When an image is captured of an image having one or more identifiers or references steganographically encoded therein, the identifiers or references may be interpreted, and the supplemental information or content may be accessed and displayed on a computer display. In some embodiments, the supplemental information or content may identify and relate to a commercial product expressed in an image, and may include a link to one or more pages or functions for purchasing the commercial product.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional of U.S. patent application Ser. No.15/078,764, filed Mar. 23, 2016, the contents of which are incorporatedby reference herein in their entirety.

BACKGROUND

A digital image is a collection of pixels, typically arranged in anarray, which defines an optically formed reproduction of one or moreobjects, backgrounds or other features of a scene. In a digital image,each of the pixels represents or identifies a color or other lightcondition associated with a portion of such objects, backgrounds orfeatures. For example, a black-and-white digital image includes a singlebit for representing a light condition of the pixel in a binary fashion(e.g., either black or white), while a grayscale digital image mayrepresent a light condition in multiple bits (e.g., two to eight bitsfor defining tones of gray in terms of percentages or shares ofblack-and-white). A color digital image may include groups of bitscorresponding to each of a plurality of base colors (e.g., red, green orblue), and the groups of bits may collectively represent a colorassociated with the pixel. One common digital image is a twenty-four bit(24-bit) color digital image, in which each of the pixels includes threechannels of eight bits each, including a first channel of eight bits fordescribing an extent of red within a pixel, a second channel of eightbits for describing an extent of green within the pixel, and a thirdchannel of eight bits for describing an extent of blue within the pixel.

Steganography is the art and science of sending and receivingcommunications in a manner that prevents their presence from beingdetected. The word “steganography,” which literally means “coveredwriting” in Greek, is commonly used to describe communicationstechniques in which invisible or subliminal messages are hidden withindigital images. A steganographic process starts by selecting contentthat is to be concealed, and by identifying portions of a base image,e.g., an image having one or more bits that may be modified to encodethe content without damaging the integrity or size of the base image.Such bits, which are sometimes referred to as “least significant bits,”“less significant bits,” or “redundant bits,” may be co-opted andreplaced with data corresponding to the content to be concealed. By itsvery nature, steganographic conversion and storage of data within a baseimage necessarily reduces the clarity and resolution of the base image,to at least a very limited extent. Where the bits of the base image areproperly selected, however, information or data may be concealed withina base image in a manner that is indiscernible to the human eye, andwhich neither increases nor decreases a total number of bits occupied bythe modified base image, thereby ensuring that a size of a fileincluding the modified base image is no larger than a size of the baseimage itself.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A through 1D are views of aspects of one system or process forsteganographic camera communication in accordance with embodiments ofthe present disclosure.

FIG. 2 is a block diagram of components of one system for steganographiccamera communication in accordance with embodiments of the presentdisclosure.

FIG. 3 is a flow chart of one process for steganographic cameracommunication in accordance with embodiments of the present disclosure.

FIGS. 4A through 4C are views of aspects of one system or process forsteganographic camera communication in accordance with embodiments ofthe present disclosure.

FIGS. 5A through 5D are views of aspects of one system or process forsteganographic camera communication in accordance with embodiments ofthe present disclosure.

FIG. 6 is a flow chart of one process for steganographic cameracommunication in accordance with embodiments of the present disclosure.

FIGS. 7A through 7D are views of aspects of one system or process forsteganographic camera communication in accordance with embodiments ofthe present disclosure.

FIGS. 8A and 8B are views of aspects of one system or process forsteganographic camera communication in accordance with embodiments ofthe present disclosure.

FIG. 9 is a view of aspects of one system or process for steganographiccamera communication in accordance with embodiments of the presentdisclosure.

FIG. 10 is a view of aspects of one system or process for steganographiccamera communication in accordance with embodiments of the presentdisclosure.

FIG. 11 is a view of aspects of one system or process for steganographiccamera communication in accordance with embodiments of the presentdisclosure.

FIG. 12 is a flow chart of one process for steganographic cameracommunication in accordance with embodiments of the present disclosure.

FIG. 13 is a view of aspects of one system or process for steganographiccamera communication in accordance with embodiments of the presentdisclosure.

FIG. 14 is a flow chart of one process for steganographic cameracommunication in accordance with embodiments of the present disclosure.

FIGS. 15A and 15B are views of aspects of one system or process forsteganographic camera communication in accordance with embodiments ofthe present disclosure.

DETAILED DESCRIPTION

As is set forth in greater detail below, the present disclosure isdirected to systems and methods for steganographic communication usingcameras. More specifically, the systems and methods of the presentdisclosure are directed to steganographically incorporating informationor data corresponding to objects, backgrounds or other features of avisual image (e.g., a digital image including color, grayscale orblack-and-white pixels, or a hard copy of such an image) into one ormore representative pixels of the visual image. The representativepixels may be modified by supplanting data stored in least significantbits or less significant bits thereof (e.g., lowest-valued bits of agrayscale pixel, or lowest-valued bits of one or more channels of acolor pixel) to generate a visual identifier associated with anysupplemental content that may relate to the visual image, e.g., to ascene represented in the visual image, or to one or more objects thatare shown, expressed or otherwise visible within the scene. The systemsand methods of the present disclosure may thus effectively co-opt one ormore bits of data indicative of a color of a representative pixel of avisual image in order to store bits of additional data indicative ordescriptive of content shown within the image, or of identifiers of(e.g., links to) indicators or descriptors of such content. Therepresentative pixels and the bits thereof in which supplemental contentis to be stored may be selected on any basis, including but not limitedto their respective locations within the visual image, or values orattributes of the color of the pixels, or at random.

In some embodiments of the present disclosure, a visual image may besteganographically encoded with one or more visual identifiers that arerepresentative of the visual image or one or more of the contentstherein. The visual identifiers may include bar codes, alphanumericcharacters (e.g., text, numbers and/or symbols) or any other type orform of symbol or other identifier that may be associated withsupplemental content of any type or form regarding the content of thevisual image. The visual identifiers are steganographically encoded intoone or more visual images in a manner that renders such identifiersinvisible to the human eye, but sufficiently detectable to imagingdevices and associated computer-implemented modules and computercomponents, thereby enabling users of such imaging devices to requestand receive supplemental content while viewing the visual images, whichmay be still or moving, and printed or rendered in any digital format.

Thus, a first image may be used to communicate additional informationregarding visual content expressed within the first image upon a requestof a user of an imaging device, e.g., by capturing a second image of thefirst image within a field of view of the imaging device, andinterpreting the second image to recognize the additional informationstored therein. The additional information may be used to identify orelaborate upon the contents of the first image, or to provide one ormore links to external resources having further information regardingsuch contents. For example, where a digital image depicts a commercialproduct, the digital image may be steganographically encoded to includetext-based information regarding the commercial product, such as a name,a manufacturer, a size, a style or a price of the commercial product, orto include one or more hyperlinks to (or identifiers of) externalnetwork pages having further details regarding the commercial product,or which offer opportunities for a customer to electronically execute apurchase of the commercial product.

Referring to FIG. 1A through 1D, views of aspects of a system or processfor steganographic camera communication in accordance with embodimentsof the present disclosure are shown. As is shown in FIG. 1A, digitalimaging data 142 features a scene and a plurality of objects, includinga bicycle 10A and one or more accessories for the bicycle 10A, such as ahelmet 10B and a pair of sunglasses 10C. The imaging data 142 may be astill image, or one or more frames of moving images, e.g., a video fileor other set of multimedia data.

The bicycle 10A, the helmet 10B and the pair of sunglasses 10C may becommercially available for purchase, such as through an onlinemarketplace or a traditional brick-and-mortar retailer. For example, asis shown in FIG. 1B, a network page 116 offering the bicycle 10A forsale at an online marketplace may include a variety of information ordata regarding the bicycle 10A, e.g., details, dimensions, ratings,costs or images of the bicycle 10A, and one or more interactive featuresby which a customer may add the bicycle 10A to a virtual shopping cartor otherwise purchase the bicycle 10A. The network page 116 may furtherinclude information or data regarding one or more items that are relatedto the bicycle 10A, such as the helmet 10B or the pair of sunglasses10C. Traditionally, in order to access the network page 116 regardingthe bicycle 10A, a customer is required to access an interactiveapplication such as a browser via one or more computer devices, andeither enter a uniform resource locator (or URL) 115 into an address baror other space provided by the browser, search for the bicycle 10A usingone or more keywords, or browse through one or more categories until heor she finds information corresponding to the bicycle 10A.

As is discussed above, in accordance with the present disclosure, visualidentifiers such as one or more references to external content may besteganographically encoded into visual imagery, in a manner that renderssuch identifiers invisible to the human eye, but readily detectable byone or more imaging devices and image processing techniques. Forexample, referring again to FIG. 1B, the URL 115 for the network page116 may be converted to a bar code 12 or other optically readableidentifier, e.g., by one or more computer devices 122 associated withthe marketplace. As is shown in FIG. 1C, the bar code 12 may then beencoded into a selected portion 143 of the image 142, such as bymodifying least significant bits or less significant bits of a patternof pixels in a central location of the image 142. For example, thealternating pattern of light and dark lines embodied in the bar code 12may be transmuted into corresponding least significant bits or lesssignificant bits in a portion of the image 142, i.e., by imperceptiblydarkening areas within the selected portion 143 of the image 142corresponding to the darkened bars of the bar code 12 and/or lighteningintervening spaces of the image 142 between the darkened bars of the barcode 12, resulting in a modified image 144 that may be stored in one ormore data stores and reproduced or displayed on demand. Thus, whereas ahuman eye could not detect the presence of the bar code 12 within themodified image 144, an imaging device that is configured to capture andprocess imaging data may readily determine not only that the bar code 12is present therein but also interpret the bar code 12 and accessinformation associated therewith, e.g., information to which the barcode 12 is linked, such as one or more of the details provided on thenetwork page 116.

As is shown in FIG. 1D, a user 130 of a smartphone 132 having one ormore imaging devices therein is viewing still or moving images on atelevision 140 or other display device. The modified image 144 isrendered on the television 140. The user 130 may align the smartphone132 in a manner that places all or a portion of the television 140 andthe modified image 144 rendered thereon within a field of view of aresident imaging device, and causes the smartphone 132 to capture someor all of the modified image 144 and the bar code 12 (not shown in FIG.1D) within the field of view. Upon recognizing the bar code 12, thesmartphone 132 may access any supplemental content associated with thebar code 12 and the URL 115, including but not limited to contentdisplayed on the network page 116 of FIG. 1B, or a modified version ofthe network page 116, in a window 14 rendered above the modified image144 on a user interface 134. As is shown in FIG. 1D, the user 130 mayrequest additional information regarding the modified image 144, or thebicycle 10A represented therein, or may close the window 14 and resumeviewing the modified image 144 or take any other action that may berelevant or desired using the smartphone 132.

Accordingly, the systems and methods of the present disclosure maysteganographically encode visual identifiers into images of any type orform, including but not limited to still images, moving images, printimages, portraits or the like. Because a steganographic process mayencode visual identifiers into images in a manner that enables thevisual identifiers to remain unnoticed by humans, but to be detected andinterpreted by computer devices, and without substantially affecting thecontent of such images, the images themselves may act as unobtrusive andoptional gateways to additional information or content relating to thecontent expressed in such images. Where a viewer of one of an imagehaving a visual identifier steganographically encoded therein intends toview additional information or content regarding the image, the viewermay capture an image of the image, e.g., using a computer deviceequipped with a camera or other imaging device, such as a smartphone ortablet computer. Once the identifier is recognized and interpreted, anexternal resource associated with the identifier may be accessed anddownloaded to the computer device, and displayed concurrently with thecaptured image of the image, or as a replacement for the captured imageof the image, on a display of the computer device, e.g., on atouchscreen of the smartphone or tablet computer. Where a viewer has nointerest in the additional information or content, however, the userneed not take any action at all, and may instead continue with his orher viewing of the image.

Imaging data in the form of visual imaging data, or depth imaging data,may be captured using one or more imaging devices such as digitalcameras, depth sensors or range cameras. Such devices may generallyoperate by capturing light that is reflected from objects, and bysubsequently calculating or assigning one or more quantitative values toaspects of the reflected light, e.g., pixels, generating an output basedon such values, and storing such values in one or more data stores.Digital cameras may include one or more sensors having one or morefilters associated therewith, and such sensors may detect informationregarding aspects of any number of pixels of the reflected lightcorresponding to one or more base colors (e.g., red, green or blue) ofthe reflected light, or distances to objects from which the light wasreflected. Such sensors may generate data files including suchinformation, and store such data files in one or more onboard oraccessible data stores (e.g., a hard drive or other like component), aswell as one or more removable data stores (e.g., flash memory devices),or displayed on one or more broadcast or closed-circuit televisionnetworks, or over a computer network as the Internet.

Imaging data files that are stored in one or more data stores may beprinted onto paper or other alternatives, presented on one or morecomputer displays, or subjected to one or more analyses, such as toidentify items expressed therein. Such data files may be stored in anynumber of formats, including but not limited to JPEG or JPG files, orGraphics Interchange Format (or “.GIF”), Bitmap (or “.BMP”), PortableNetwork Graphics (or “.PNG”), Tagged Image File Format (or “.TIFF”)files, Audio Video Interleave (or “.AVI”), QuickTime (or “.MOV”), MovingPicture Experts Group (or “.MPG,” “.MPEG” or “.MP4”) or Windows MediaVideo (or “.WMV”) files.

Reflected light may be captured or detected by an imaging device if thereflected light is within the device's field of view, which is definedas a function of a distance between a sensor and a lens within thedevice, viz., a focal length, as well as a location of the device and anangular orientation of the device's lens. Accordingly, where an objectappears within a depth of field, or a distance within the field of viewwhere the clarity and focus is sufficiently sharp, an imaging device maycapture light that is reflected off objects of any kind to asufficiently high degree of resolution using one or more sensorsthereof, and store information regarding the reflected light in one ormore data files.

Many imaging devices also include manual or automatic features formodifying their respective positions, fields of view or orientations.For example, a digital camera may be configured in a fixed position, orwith a fixed focal length (e.g., fixed-focus lenses) or angularorientation. Alternatively, an imaging device may include one or moreactuated or motorized features for adjusting a position of the imagingdevice, or for adjusting either the focal length (e.g., zooming theimaging device) or the angular orientation (e.g., the roll angle, thepitch angle or the yaw angle), by causing a change in the distancebetween the sensor and the lens (e.g., optical zoom lenses or digitalzoom lenses), a change in the location of the imaging device, or achange in one or more of the angles defining the angular orientation.

For example, an imaging device may be hard-mounted to a support ormounting that maintains the device in a fixed configuration or anglewith respect to one, two or three axes. Alternatively, however, animaging device may be provided with one or more motors and/orcontrollers for manually or automatically operating one or more of thecomponents, or for reorienting a position, axis or direction of thedevice, i.e., by moving, panning or tilting the device. Panning animaging device may cause a rotation within a horizontal plane or about avertical axis (e.g., a yaw), while tilting an imaging device may cause arotation within a vertical plane or about a horizontal axis (e.g., apitch). Additionally, an imaging device may be rolled, or rotated aboutits axis of rotation, and within a plane that is perpendicular to theaxis of rotation and substantially parallel to a field of view of thedevice.

Some modern imaging devices may digitally or electronically adjust animage identified in a field of view, subject to one or more physical andoperational constraints. For example, a digital camera may virtuallystretch or condense the pixels of an image in order to focus or broadenthe field of view of the digital camera, and also translate one or moreportions of images within the field of view. Imaging devices havingoptically adjustable focal lengths or axes of orientation are commonlyreferred to as pan-tilt-zoom (or “PTZ”) imaging devices, while imagingdevices having digitally or electronically adjustable zooming ortranslating features are commonly referred to as electronic PTZ (or“ePTZ”) imaging devices.

Information and/or data regarding features or objects expressed inimaging data, including colors, textures or outlines of the features orobjects, may be extracted from the data in any number of ways. Forexample, colors of pixels, or of groups of pixels, in a digital imagemay be determined and quantified according to one or more standards,e.g., the RGB (“red-green-blue”) color model, in which the portions ofred, green or blue in a pixel are expressed in three correspondingnumbers ranging from 0 to 255 in value, or a hexadecimal model, in whicha color of a pixel is expressed in a six-character code, wherein each ofthe characters may have a range of sixteen. Colors may also be expressedaccording to a six-character hexadecimal model, or # NNNNNN, where eachof the characters N has a range of sixteen digits (i.e., the numbers 0through 9 and letters A through F). The first two characters NN of thehexadecimal model refer to the portion of red contained in the color,while the second two characters NN refer to the portion of greencontained in the color, and the third two characters NN refer to theportion of blue contained in the color. For example, the colors whiteand black are expressed according to the hexadecimal model as # FFFFFFand #000000, respectively, while the color candy apple red is expressedas # D61123. Any means or model for quantifying a color or color schemawithin an image or photograph may be utilized in accordance with thepresent disclosure. Moreover, textures or features of objects expressedin a digital image may be identified using one or more computer-basedmethods, such as by identifying changes in intensities within regions orsectors of the image, or by defining areas of an image corresponding tospecific surfaces.

Edges, contours, outlines, colors, textures, silhouettes, shapes orother characteristics of objects, or portions of objects, expressed instill or moving digital images may be identified using one or morealgorithms or machine-learning tools. The objects or portions of objectsmay be stationary or in motion, and may be identified at single, finiteperiods of time, or over one or more periods or durations. Suchalgorithms or tools may be directed to recognizing and markingtransitions (e.g., the edges, contours, outlines, colors, textures,silhouettes, shapes or other characteristics of objects or portionsthereof) within the digital images as closely as possible, and in amanner that minimizes noise and disruptions, and does not create falsetransitions. Some detection algorithms or techniques that may beutilized in order to recognize characteristics of objects or portionsthereof in digital images in accordance with the present disclosureinclude, but are not limited to, Canny edge detectors or algorithms;Sobel operators, algorithms or filters; Kayyali operators; Roberts edgedetection algorithms; Prewitt operators; Frei-Chen methods; or any otheralgorithms or techniques that may be known to those of ordinary skill inthe pertinent arts.

Steganography is typically defined as the art, or the science, ofembedding or concealing information or data within other information ordata. In a computer-based context, steganographic techniques arecommonly utilized to embed one or more bits of data, or streams of suchbits, into one or more other bits of data, or one or more other streamsof such bits. For example, information or data is commonlysteganographically encoded into imaging data, e.g., files including oneor more still or moving images, without altering the information or databeing encoded, and without substantially altering the imaging data to anappreciable degree.

To date, steganography has been extensively and nearly exclusively usedfor encrypting hidden information or metadata into digital files, suchas images. For example, fraudsters, spammers or hackers are known tosteganographically inject executable files into seemingly harmlessimages, and to send one or more of the steganographically altered imagesvia electronic mail or other messaging techniques to a computer deviceof an unsuspecting recipient. Steganography is also believed to havebeen used in a number of military and law enforcement applications, aswell.

According to some steganographic techniques, information or data may beencoded into one or more least significant bits or bytes of an image (orless significant bits or bytes of the image). For example, in a 24-bitdigital image with pixels having three eight-bit channels (e.g., a redchannel, a green channel and a blue channel), the available capacity forstoring information in each pixel is substantial. Because each of thethree eight-bit channels in a pixel may have two hundred fifty-six(i.e., two to the eighth power, 2⁸) unique values, each pixel of athree-channel 24-bit digital image may represent one of 16,777,216(e.g., two to the eighth power cubed, (2⁸)³; or two hundred fifty-sixcubed, 256³) unique colors. Therefore, some steganographic techniquesoperate by identifying one or more least significant bits or bytes, orless significant bits or bytes, in one or more of the channels of apixel, and replacing one or more of such least significant bits or byteswith information or data, without causing any significant degradation ofthe clarity or resolution of the image.

For example, by storing information or data within a single leastsignificant bit of each of three color channels in a pixel, the numberof unique colors that may be represented in the pixel is reduced to2,097,152 (e.g., by one-eighth). In an image of substantially highresolution, the reduction in the number of available colors that may berepresented using the remaining seven bits of each of the three channelsof the pixel is typically indiscernible to the human eye. However, sucha reduction provides three bits for storing additional information ordata within each of the pixels of the 24-bit digital image. In astandard 3.1 megapixel (MP) digital image having dimensions of 2048×1536pixels, nearly nine-and-one-half million bits, or 1.2 megabytes (MB) ofstorage may be so provided.

Similarly, steganographically storing information or data within twoleast significant bits of each of the three color channels in a 24-bitdigital image pixel reduces the maximum number of unique colors that maybe represented in such pixels to 262,144 (e.g., by one-sixty-fourth, ortwo to the negative sixth power, 2⁻⁶), which is also likelyindiscernible to the human eye, and yet may create up to an additional2.4 megabytes (MB) of storage capacity within a standard 3.1 megapixeldigital image, without typically increasing a size of the file.Likewise, steganographically storing information or data within threeleast significant bits of each of the three color channels in a 24-bitdigital image pixel reduces the number of unique colors that may berepresented in such pixels to 32,768 (e.g., by one-five-hundred-twelfth,or two to the negative ninth power, 2⁻⁹), which is also likelyindiscernible to the human eye, but may create up to an additional 3.5megabytes (MB) of storage capacity within a standard 3.1 megapixeldigital image, without typically increasing a size of the file.

Thus, steganography and steganographic techniques may require an imageto sacrifice clarity or resolution (e.g., color quality in visualimaging data, or depth precision in depth imaging data), but may, inexchange for this sacrifice, provide an internal storage means withinthe image to store one or more bits of information or data. Where thenumber of bits of information or data to be encoded into an image issmall, or may be spread out across the image, the loss in clarity orresolution resulting from this sacrifice is insubstantial and may gounnoticed by humans.

The systems and methods of the present disclosure are directed toutilizing steganographic techniques to encode visual identifiers, orlinks, into visual imaging data (e.g., color imaging data, well asblack-and-white or grayscale imaging data). The identifiers or links maybe associated with any type or form of external content, e.g.,text-based information, audio signals, video signals or otherinformation or data. Thus, once an imaging device captures one or moreimages of an image that has been steganographically encoded with one ormore visual identifiers or links to external content, such as themodified image 144 shown in FIG. 1D, the identifiers or links may berecognized and interpreted, and used to access the external content. Auser may operate any type of imaging device provided in association witha computing device in order to capture imaging data regardingsteganographically encoded images. For example, such imaging data may becaptured using any type or form of imaging device such as a red, green,blue (“RGB”) color camera, a still camera, a motion capture/videocamera, as well as imaging devices or other devices that are configuredto capture depth information or data, including depth-sensing camerassuch as RGBz (or RGBD) cameras. In some other implementations, one ormore of the cameras may be a thermographic or infrared (IR) camera.

A visual identifier may be steganographically encoded into a visualimage in any manner that enables the identifier to remain undetected bythe human eye yet enhances the visibility of the identifier to imagingdevices or other optical machines. Human eyes and their visual axes maybe forcibly repositioned by large-scale movements of the head or neck,or by smaller movements of the eyes, known as saccades or saccadic eyemovements, which redirect a fovea of a retina from one point of interestto another. By some estimates, saccades occur in most humans three tofour times per second. Thus, whereas a human eye may provide a visualfield of view of approximately two hundred degrees (200°), the human eyetypically captures intensely detailed information from a fovea in theform of a slender band having a width of a handful of degrees. The foveaincludes tens of thousands of photoreceptors that change their positionevery two hundred to three hundred milliseconds. In contrast, an imagingdevice may capture imaging data for each of a plurality of image pixelswithin an entire field of view upon a shuttering, with the field of viewbeing defined by the properties of a lens and not by any involuntarynarrowing thereof, as with a fovea. Thus, the image pixels of the entirefield of view may be processed, e.g., by one or more computerprocessors, to recognize any color variations or shading variationsexpressed therein, to determine whether such color variations or shadingvariations are consistent with one or more visual identifiers, and tointerpret such visual identifiers and/or to identify any relevantsupplemental information associated with such identifiers.

The systems and methods for steganographic communication disclosedherein may be further enhanced by physical or virtual phenomena that arepresent when a first set of a second set of imaging data (e.g., one ormore still or moving images) is captured by an imaging device with otherimaging data (e.g., one or more other still or moving images) renderedwithin a field of view of the imaging device. Where a digital camera isaligned to capture still or moving images of a video display devicehaving still or moving images rendered thereon, differences between aframe rate of the digital camera and a refresh rate of the video displaydevice may cause one or more patterns of interference, sometimes calledMoiré patterns, to visibly alter the manner in which the rendered stillor moving images appear within the captured still or moving images. Suchinterference effects may accentuate a presence of a visual identifierwithin a captured image of a displayed image, thereby increasing alikelihood that the visual identifier will be recognized and interpretedby an imaging device and one or more associated computer processors.Moreover, similar effects may be observed where the imaging device thatis aligned to capture imaging data from a displayed image, and thedisplayed image itself, are also in relative motion.

The systems and methods of the present disclosure permit imaging systemsor methods to capture one or more images of other images and to rapidlydetermine information pertaining to contents of such other images basednot on a direct analysis of such contents but by interpreting visualidentifiers that are steganographically encoded therein, yet areinvisible to the human eye. Accordingly, such systems and methods mayrecognize and interpret information regarding such contents more rapidlyand efficiently than traditional imaging systems or methods, and in waysthat such imaging systems or methods cannot emulate. For example, wherean image depicts a man drinking a clear beverage or wearing a wool suit,at best, traditional imaging systems or methods may be able to determinethat the image includes an outline of a man, or that the man is holdinga glass, e.g., using one or more edge detection or color or textureanalysis techniques. Even if such systems or methods are able torecognize the man or the glass, however, no such system or method maydetermine whether the clear beverage is water, vinegar, soda ormedicine, let alone whether the water is tap water or spring water,whether the vinegar or the soda are of a particular type or brand, orwhether the medicine is of a specific strength or concentration.Likewise, such systems are unable to determine a material, a brand, asize, a style or a manufacturer of the suit, or locations or identitiesof merchants or vendors from which the suit may be purchased. Thesystems and methods of the present disclosure enable supplementalcontent regarding images, or links to supplemental content, to besteganographically encoded within such images, and enable viewers ofsuch images to access the supplemental content at their own discretion.

In accordance with the present disclosure, visual identifiers that aresteganographically encoded into imaging data may be of any type or form,and may have any dimension. For example, one steganographically encodedvisual identifier may be a bar code (e.g., a one-dimensional ortwo-dimensional bar code, such as a QR code) that may be specificallyencoded to embody or represent supplemental information or content, or alink to supplemental information or content. Another steganographicallyencoded visual identifier may be a set of alphanumeric characters,including one or more letters or numbers. For example, visual imagingdata may be steganographically encoded with text including thesupplemental information or content itself, which may be recognized inone or more images captured from the imaging data. Furthermore, thealphanumeric characters may embody one or more URLs associated with anexternal resource, e.g., a network page or other set of networked data,and such characters may be recognized and used to access the externalresource. Yet another steganographically encoded visual identifier maybe a polygon or other shape or symbol. Any type of information or datathat may be encoded into a digital image by creating a contrast betweenand among pixels that may be discerned by an imaging device having oneor more computer processors, but remains substantially invisible to thehuman eye, may act as a visual identifier in accordance with the presentdisclosure.

Those of ordinary skill in the pertinent arts will recognize that visualidentifiers may be steganographically encoded into visual imaging dataat any time after the imaging data has been captured, e.g., immediatelyafter the imaging data has been captured, or at any other date or time,and by anyone in possession or control of the imaging data and/or thecontent therein. For example, one or more visual identifiers may beencoded into imaging data in real time or in near-real time, e.g., intoone or more frames of moving images captured in a “live” format, or atany subsequent time. Alternatively, the one or more visual identifiersmay be encoded into a still image, also in real time or in near-realtime, or at any subsequent time. Moreover, once one or more visualidentifiers have been encoded into imaging data, the modified imagingdata may be made available to one or more potential viewers immediately,such as by presenting the modified imaging data on one or more displays,e.g., in still or moving form, or by printing the imaging data on paperor other media. Thereafter, one or more imaging devices (e.g., a mobiledevice such as a smartphone or tablet computer) may capture an image ofthe modified imaging data, and the one or more visual identifiersencoded therein may be recognized and interpreted in order to causesupplemental information or content relating to the modified imagingdata to be rendered thereby. Additionally, representative pixels ofimaging data may be modified to encode a visual identifier therein, andmodified again to remove the visual identifier therefrom, therebyrestoring the imaging data to a virgin state. Subsequently, the imagingdata may be modified again to encode a new visual identifier therein.The process of encoding visual identifiers into imaging data andremoving the visual identifiers therefrom may be repeated as many timesas is necessary or desired.

A visual identifier such as a bar code or a set of alphanumericcharacters may be steganographically encoded into visual imaging data byselecting a location within one or more images or frames of the visualimaging data that may accommodate the visual identifier, and defining apattern of pixels that must be steganographically altered in order tocreate a visual contrast that may be recognized by an imaging device andone or more computers, but not by the human eye. Some visual identifiersmay include one-dimensional bar codes, two-dimensional bar codes,bokodes or any other form of coded representations of information ordata. Some common one-dimensional bar codes include Code 39 codes,Extended Code 39 codes, Code 128 Codes, UCC/European Article Numbering(or “EAN”) 128 codes, Universal Product Code (or “UPC”) A or E codes,EAN13 or EAN8 codes, Interleaved (or “ITF”) codes, CodaBar codes, Code11 codes, Code 93 codes, GS1 DataBar codes, GS1 DataBar Expanded codes,GS1 DataBar Limited codes, GS1 DataBar Coupon codes, DataBarOmni-Directional Stacked or Truncated codes, MSI Plessey codes, POSTNETcodes, PLANET codes or OneCode codes. Some common two-dimensional barcodes include PDF417 codes, Data Matrix codes, MaxiCode codes, Azteccodes or QR codes. Additionally, some visual identifiers may alsoinclude other alphanumeric characters or symbols.

In some embodiments, where a visual identifier to be encoded into visualimaging data includes a bar code having a pattern of alternating lightand dark sections, a section of one or more frames of the visual imagingdata that is sufficiently sized to accommodate the pattern may beselected, and regions of pixels may be steganographically altered tocreate an appropriate level of contrast between them. For example, wherea digital image includes a section of high-resolution blaze orangehaving red, green and blue color values of (246, 103, 51) in decimal,one or more portions of the digital image corresponding to dark bars orsections of a bar code may be modified by converting values of lesssignificant bits to create a slightly darker orange, while one or moreother portions of the digital image corresponding to light bars orsections of the bar code may be modified by converting values of theless significant bits to create a slightly lighter orange. Byselectively altering pixels within adjacent sections of a digital image,a visual identifier comprised of a bar code having different colors incontrast that are imperceptible to humans but may be readily discernedby imaging devices may be embedded into the digital image thereby.

For example, whereas the red color value of the blaze orange is 246 indecimal, or 11110110 in binary, the red color value may be varied simplyby changing one or more of the final three binary digits of the redcolor value (e.g., by increasing or decreasing the digits) to within arange of 11110000 to 11110111, which corresponds to red color values of240 to 248 in decimal. Similarly, the green and blue color values of 103and 51 in decimal, or 01100111 and 00110011 in binary, respectively, ofthe blaze orange may be varied simply by changing one or more of thefinal three binary digits of the respective color values to withinranges of 01100000 to 01100111 in binary, or 96 to 103 in decimal, forthe green color value and ranges of 00110000 to 00110111 in binary, or48 to 55 in decimal, for the blue color value, respectively. Thus, thevisual identifier may be expressed in the section of the orange bycontrast between a first variant of orange and a second variant orangewithin a section of blaze orange such that the visual identifier isimperceptible to human viewers, but may be readily discerned by one ormore imaging devices having computer processors associated therewith.The synthesis of visual contrast within a digital image bysteganographic modification of least significant bits or lesssignificant bits may be used to encode any type of visual identifierwithin the digital image, including not only bar codes but any type orform of alphanumeric characters, symbols or other markings or indicia.

A visual identifier may be steganographically encoded into any relevantor appropriate portion of an image (e.g., a spatial location within theimage). According to some embodiments of the present disclosure,representative pixels for encoding the visual identifier may becentrally located within an image frame, e.g., generally across all or aportion of the image frame. In some other embodiments, however, therepresentative pixels may be located in a standard location within animage frame (e.g., a perimeter of the frame, a designated corner of theframe, or any other section of the frame), and an imaging device thatcaptures one or more images of the frame may be configured to recognizethat the image includes an outline of an image, and to search thestandard location within the outline of the image for one or more visualidentifiers.

In addition to spatial variations, visual indicators may also be variedtemporally within imaging data. For example, as is discussed above, abar code, a set of alphanumeric characters or one or more otheridentifiers may be placed in any location within a visual image.Likewise, different visual images may be encoded into discrete frames ofimaging data, e.g., portions of a video file, and configured to appearat different times when the imaging data is played. In some embodiments,a series of video frames including a plurality of objects within a scenemay be encoded to include visual identifiers corresponding to the sceneand/or one or more of the objects simultaneously, or at different times,as the video frames are played on a display device. For example, a firstvisual identifier associated with general information descriptive of thescene may be displayed at the outset, e.g., as an introduction, andsubsequent visual identifiers may be displayed at later times, such aswhen specific objects enter the scene or are featured prominently.

Moreover, in order to facilitate a search for one or more visualidentifiers within a visual image, a selected visual identifierindicating whether the visual image includes one or more other visualidentifiers, or where such identifiers are located within the visualimage, may be encoded therein, e.g., within a predetermined location inthe visual image. The selected visual identifier may be a “telltaleindicator” of the presence or absence of other visual identifiers ofadditional information, or locations of the additional information, andneed not itself link to or be associated with any additionalinformation. Thus, when an image of the visual image is captured, onlythe predetermined location need be searched for visual identifiers. If atelltale indicator is found therein, the telltale indicator may beinterpreted to determine whether the visual image includes one or moreother visual identifiers, or where such identifiers may be located. Forexample, the selected visual identifier or telltale indicator may be asymbol, a character, an icon or any other feature, or one or more ofsuch features, which may be steganographically encoded into a visualimage.

According to some embodiments of the present disclosure, representativepixels for encoding one or more visual identifiers into a digital imagemay be selected according to methods or techniques that systematicallyevaluate pixels within the visual imaging data (e.g., color, grayscaleor black-and-white pixels) and identifies and selects representativepixels for storing steganographically encoded visual identifierstherein. For example, the representative pixels may be identified atrandom, or in specific locations within visual imaging data (e.g.,predefined locations within the images, such as in a grid or accordingto a pattern, or based on a rectangular or polar coordinate systemcentered or based in one or more specific locations of the imagingdata). Such methods or techniques may identify one or more homogenoussets or subsets of representative pixel regions or sectors (e.g., gridsor groupings of pixels) of visual imaging data, one or more of which maybe ultimately selected for encoding with pixels corresponding to avisual identifier.

In some other embodiments of the present disclosure, representativepixels may be identified in locations that may be selected based onattributes of the visual imaging data. For example, representativepixels may be identified or defined within visual imaging data as afunction of variations of pixel-level intensities within the visualimaging data, which may be quantitatively determined by recognizingvarious texture properties within the visual imaging data, comparingnumerical measures of the relative intensities of pixels or textonswithin the visual imaging data to one another, or on any other basis.Representative pixels may be selected based on one or more repetitivepatterns observed within visual imaging data, i.e., locations within theimaging data where the properties of the imaging data are constant,varying slowly, or substantially periodic in nature, and may berecognized using one or more algorithms or machine-learning tools. Anymeans or model for selecting representative pixels from a set of visualimaging data (e.g., a still or moving black or white, grayscale, coloror depth image captured from a scene) for steganographically encodingvisual identifiers therein may be utilized in accordance with thepresent disclosure.

Once representative pixels have been identified and/or selected forsteganographically encoding a visual identifier therein, the type ofvisual identifier may be selected, and a pattern of pixels correspondingto the visual identifier may be defined and encoded into correspondingpixels within a color image or other set of visual imaging data to forma steganographically encoded image. Such bits may be steganographicallyencoded into one or more of the least significant color, grayscale orblack-and-white bits of channels of pixels of visual imaging data (e.g.,bits of a single channel of a grayscale imaging data, or bits ofmultiple channels corresponding to base colors within the imaging data)without significantly impacting the overall quality of the visualimaging data.

For example, in a 24-bit representative pixel of visual imaging data,some of the least significant bits of each of the color channels may beco-opted and modified in order to encode a visual identifier within suchpixels. Using six bits of data, for example, a visual identifier may beembedded into two bits of each of three channels (e.g., red, green andblue) of visual imaging data. As is discussed above, when using two bitsof each of the three channels in a region of representative color pixelsto generate a contrast for storing visual identifiers therein, the colorvariability in the region of pixels drops from 16,777,216 to 262,144, orby one sixty-fourth, a difference that may not be readily discerned bythe human eye, even when the region is viewed independent from othercolor pixels within the visual image. When the region including themodified pixels is surrounded by regions of tens, hundreds or eventhousands of other color pixels that have not been so altered, however,the difference in color quality between the representative color pixeland the other color pixels is functionally insignificant. Those ofordinary skill in the pertinent arts will recognize that imaging datahaving pixels of more than twenty-four bits (e.g., thirty bits,thirty-six bits, forty-eight bits, two hundred fifty-six bits), or fewerthan twenty-four bits, may be utilized to generate a steganographicallyencoded image having visual identifiers therein.

Therefore, in accordance with the present disclosure, when evaluatingthe pixels of a segmented region or sector in order to identify one ormore representative pixels into which a visual identifier may beencoded, the systems and methods disclosed herein may consider factorsor variables pertaining to the variances in image noise such assignal-to-noise ratios or other related factors or variables. Forexample, a predetermined threshold for a pixel variance orsignal-to-noise ratio may be determined for each of a plurality ofsegmented regions or sectors, or for a given segmented region or sector,and all pixels having variances below a first threshold, orsignal-to-noise ratios above a second threshold, may be excluded fromconsideration as representative pixels.

After a visual identifier has been encoded into representative pixels ofa visual imaging data file, the modified visual imaging data file may bestored in one or more data stores, and utilized for any purpose. Forexample, the representative pixels having the visual identifier encodedtherein may be identified from a stored modified visual imaging datafile according to the same strategy by which the representative pixelswere originally identified (e.g., at random, according to one or morepatterns, or based on attributes of the depth information or data or thevisual imaging data). The visual identifier may then be recognized andused to access supplemental information or content, e.g., directly orfrom an external resource, and utilized for any purpose. For example, asis shown in FIGS. 1B through 1D, supplemental information or dataassociated with the bicycle 10A (e.g., all or portions of data providedon the network page 116) may be accessed and presented to the user 130on the user interface 134, e.g., in the window 14, once thesteganographically encoded bar code 12 is recognized by the smartphone132.

Those of ordinary skill in the pertinent arts will recognize thatreferences to a “least significant bit” herein may refer to not onlylowest-valued bits of a pixel (e.g., ones-place bits) but also one ormore other bits of comparatively low significance, or “less significantbits,” including those bits that may immediately follow or have anext-highest significance above a lowest-valued bit in a given pixel.For example, in accordance with the present disclosure, one, two, threeor more bits of a representative pixel of a visual image (e.g., a color,grayscale or black-and-white pixel) may be modified to store a visualidentifier within the visual image. Accordingly, those of ordinary skillin the pertinent arts will recognize that the term “least significantbit,” as used herein, may refer to a single bit of a pixel having alowest value of all bits in the pixel, and may also refer to two or morebits of the pixel having the lowest values of all bits in the pixel.

Referring to FIG. 2, a block diagram of components of one system 200 forsteganographic camera communication in accordance with embodiments ofthe present disclosure is shown. Except where otherwise noted, referencenumerals preceded by the number “2” shown in the block diagram of FIG. 2indicate components or features that are similar to components orfeatures having reference numerals preceded by the number “1” shown inFIGS. 1A through 1D.

The system 200 of FIG. 2 includes a marketplace 210, a mediadistribution facility 220 and a user 230 (e.g., a customer) havingdevices that may be connected to one another over a network 280. Themarketplace 210 may be any entity or individual that wishes to makeitems from a variety of sources (e.g., vendors, manufacturers, merchantsor sellers) available for download, purchase, rent, lease or borrowingby customers using a networked computer infrastructure, including one ormore physical computer servers 212 and databases 214 (or data stores)for hosting a network site 216 (or network sites). The marketplace 210may be physically or virtually associated with one or more storage ordistribution facilities, such as a fulfillment center, and/or with oneor more vendors, manufacturers, merchants or sellers (not shown). Thenetwork site 216 may be implemented using the one or more servers 212,which connect or otherwise communicate with the one or more databases214 as well as the network 280, as indicated by line 218, through thesending and receiving of digital data. Moreover, the databases 214 mayinclude any type of information regarding items that have been madeavailable for sale through the marketplace 210, or ordered by customers,such as the customer 230, from the marketplace 210, or any informationor data regarding deliveries of such items to the customers.

The media distribution facility 220 may be any source or origin ofdigital media, e.g., still or moving images or other video content,audio content or other multimedia by way of a networked computerinfrastructure, including one or more physical computer servers 222 anddatabases 224 (or data stores) for hosting a network site 226 (ornetwork sites). For example, the media distribution facility 220 of FIG.2 may be provided in connection with one or more physical or virtualservices configured to manage or monitor such files, as well as one ormore other functions. The servers 222 may be connected to or otherwisecommunicate with the databases 224 and the network site 226. Thedatabases 224 may store any type of information or data, includingdigital media files or any like files containing multimedia (e.g., audioand/or video content), for any purpose. The servers 222 and/or thenetwork site 226 may also connect to or otherwise communicate with thenetwork 280, as indicated by line 228, through the sending and receivingof digital data.

In some embodiments, the media distribution facility 220 may be anInternet-based streaming content and/or media service provider. In someother embodiments, the media distribution facility 220 may be a mediasharing system. In still other embodiments, the media distributionfacility may be a radio or television station (e.g., for over-the-air,cable or Internet-based broadcasts) configured to distribute the mediavia any number of wired or wireless connections. For example, the mediadistribution facility 220 may be configured to distribute media over thenetwork 280 to one or more general purpose computers or computers thatare dedicated to a specific purpose. The media distribution facility 220may also be configured to transmit content via a direct broadcast system260, e.g., a satellite-based distribution system, over the line 262,which may be a wired or wireless connection. The media distributionfacility 220 may further be configured to transmit content to one ormore specifically configured components such as set-top boxes 250 orlike units or components (e.g., cable boxes or converters), over theline 252, which may be a wired or wireless connection. The set-top boxes250 may be configured to receive content from one or more other sources,in addition to the media distribution facility 220, such as from one ormore networked computers via the network 280, as indicated by line 258.

The user 230 may be any individual having access to one or more computerdevices or general purpose or special purpose devices for viewingcontent and/or communicating with other computer devices. For example,the user may operate a computer 232 and/or a media display device 240.The computer 232 may be any type of networked computer device (e.g., apersonal digital assistant, a digital media player, a digital mediaplayer, a smartphone, a web pad, an electronic book reader, a desktopcomputer, a laptop computer or a tablet computer, as well as a wearablecomputer device such as a pair of augmented reality glasses or awristwatch, or a computer device that may be incorporated into one ormore vehicles or appliances) or any other like machine that may operateor access one or more software applications, such as a browser or ashopping application, via a user interface 234, and may be connected toor otherwise communicate with the marketplace 210, the mediadistribution facility 220 or any other external computer devices (notshown) through the network 280, as indicated by line 238, by thetransmission and receipt of digital data.

In particular, the computer 232 may include one or more onboard orassociated imaging devices provided thereon or therewith. For example,the computer 232 may include one or more imaging devices having RGBsensors, depth sensors, memory or storage components, processors or anyother features that may be required in order to capture, analyze and/orstore imaging data from within an environment, including but not limitedto images or other content rendered by the media display device 240. Forexample, the computer 232 and any imaging devices or sensors providedthereon may be configured to capture one or more still or moving images,along with any relevant audio signals or other information. The imagingdevices provided on or in association with the computer 232 may be anytype or form of system component for capturing imaging data (e.g.,reflected light) of any kind or for any purpose. For example, in someembodiments, such imaging devices may include a red, green, blue (“RGB”)color camera, a still camera, a motion capture/video camera or any othertype or form of camera. In other embodiments, such imaging devices mayinclude a depth-sensing camera, such as an RGBz (or RGBD) camera. Instill other embodiments, such imaging devices may include athermographic or infrared (IR) camera. Additionally, in someembodiments, such imaging devices may include a camera module includinga lens and an image sensor configured to convert an optical imageobtained by the lens of the camera into a digital signal or digitalrepresentation of the image, including image resolutions of varyingdegrees that may be captured and stored at various rates (e.g., framesper second).

Although the system 200 shown in FIG. 2 includes a single user 230having a single computer 232, and a single media display device 240,those of ordinary skill in the pertinent arts will recognize that anynumber of computer devices having any number or type of imaging devicesthereon or therein, and any number of media display devices 240, may beprovided within a given environment in accordance with the presentdisclosure.

The media display device 240 may be a television, a monitor or any otherlike machine having a screen for viewing rendered video content. Forexample, the media display device 240 may incorporate any number ofactive or passive display technologies or systems, including but notlimited to electronic ink, liquid crystal displays (or “LCD”),light-emitting diode (or “LED”) or organic light-emitting diode (or“OLED”) displays, cathode ray tubes (or “CRT”), plasma displays,electrophoretic displays, image projectors, or other display mechanismsincluding but not limited to micro-electromechanical systems (or“MEMS”), spatial light modulators, electroluminescent displays, quantumdot displays, liquid crystal on silicon (or “LCOS”) displays,cholesteric displays, interferometric displays or others. As is shown inFIG. 2, the media display device 240 may be configured to receivecontent from any number of sources via one or more wired or wirelessconnections, including but not limited to the direct broadcast system260, via the line 264; the set-top box 250, via the line 254, or one ormore computer devices over the network 280, via the line 248.

Those of ordinary skill in the pertinent arts will recognize that themedia display 240 may include any number of hardware components oroperate any number of software applications for receiving and renderingcontent from the media distribution facility 220 via the directbroadcast system 260 and/or the set-top box 250, including any number oftransceivers or like devices. Moreover, those of ordinary skill in thepertinent arts will further recognize that the components or functionsof the set-top box 250 may be performed by or within the media displaydevice. Alternatively, in some embodiments, the media display device 240need not be associated with the user 230. For example, the media displaydevice 240 may be provided in a public place, beyond the control of theuser 230, e.g., a television provided in a bar, restaurant, transitstation, or shopping center, or an electronic billboard provided in apopulation center or along a transit line. In some other embodiments,the media display device 240 may be a billboard, a frame, an easel orany other system configured for displaying a print image that issteganographically encoded with one or more visual identifiers, and neednot be connected to any type or form of network or interactive system,or operate under electrical power.

The network 280 may be any wired network, wireless network, orcombination thereof, and may comprise the Internet in whole or in part.In addition, the network 280 may be a personal area network, local areanetwork, wide area network, cable network, satellite network, cellulartelephone network, or combination thereof. The network 280 may also be apublicly accessible network of linked networks, possibly operated byvarious distinct parties, such as the Internet. In some embodiments, thenetwork 280 may be a private or semi-private network, such as acorporate or university intranet. The network 280 may include one ormore wireless networks, such as a Global System for MobileCommunications (GSM) network, a Code Division Multiple Access (CDMA)network, a Long Term Evolution (LTE) network, or some other type ofwireless network. Protocols and components for communicating via theInternet or any of the other aforementioned types of communicationnetworks are well known to those skilled in the art of computercommunications and thus, need not be described in more detail herein.

The computers, servers, devices and the like described herein have thenecessary electronics, software, memory, storage, databases, firmware,logic/state machines, microprocessors, communication links, displays orother visual or audio user interfaces, printing devices, and any otherinput/output interfaces to provide any of the functions or servicesdescribed herein and/or achieve the results described herein. Also,those of ordinary skill in the pertinent art will recognize that usersof such computers, servers, devices and the like may operate a keyboard,keypad, mouse, stylus, touch screen, or other device (not shown) ormethod to interact with the computers, servers, devices and the like, orto “select” an item, link, node, hub or any other aspect of the presentdisclosure.

The marketplace server 212, the media distribution facility server 222,the user computer 232, the user media display device 240, the set-topbox 250 and/or the direct broadcast system 260 may use any web-enabledor Internet applications or features, or any other client-serverapplications or features including E-mail or other messaging techniques,to connect to the network 280, or to communicate with one another, suchas through short or multimedia messaging service (SMS or MMS) textmessages. For example, the servers 212, 222, the computer 232, the mediadisplay device 240, the set-top box 250 and/or the direct broadcastsystem 260 may be configured to transmit information or data in the formof synchronous or asynchronous messages to one another in real time orin near-real time, or in one or more offline processes, via the network280. Those of ordinary skill in the pertinent art would recognize thatthe marketplace 210, the media distribution facility 220 or the user 230may operate any of a number of computing devices that are capable ofcommunicating over the network 280. The protocols and components forproviding communication between such devices are well known to thoseskilled in the art of computer communications and need not be describedin more detail herein.

The data and/or computer executable instructions, programs, firmware,software and the like (also referred to herein as “computer executable”components) described herein may be stored on a computer-readable mediumthat is within or accessible by computers or computer components such asthe servers 212, 212 or the computer 232, or to any other computers orcontrol systems utilized by the marketplace 210, the media distributionfacility 220 or the user 230, or by or in association with the directbroadcast system 260 and/or the set-top box 250, and having sequences ofinstructions which, when executed by a processor (e.g., a centralprocessing unit, or “CPU”), cause the processor to perform all or aportion of the functions, services and/or methods described herein. Suchcomputer executable instructions, programs, software and the like may beloaded into the memory of one or more computers using a drive mechanismassociated with the computer readable medium, such as a floppy drive,CD-ROM drive, DVD-ROM drive, network interface, or the like, or viaexternal connections.

Some embodiments of the systems and methods of the present disclosuremay also be provided as a computer-executable program product includinga non-transitory machine-readable storage medium having stored thereoninstructions (in compressed or uncompressed form) that may be used toprogram a computer (or other electronic device) to perform processes ormethods described herein. The machine-readable storage media of thepresent disclosure may include, but is not limited to, hard drives,floppy diskettes, optical disks, CD-ROMs, DVDs, ROMs, RAMs, erasableprogrammable ROMs (“EPROM”), electrically erasable programmable ROMs(“EEPROM”), flash memory, magnetic or optical cards, solid-state memorydevices, or other types of media/machine-readable medium that may besuitable for storing electronic instructions. Further, embodiments mayalso be provided as a computer executable program product that includesa transitory machine-readable signal (in compressed or uncompressedform). Examples of machine-readable signals, whether modulated using acarrier or not, may include, but are not limited to, signals that acomputer system or machine hosting or running a computer program can beconfigured to access, or including signals that may be downloadedthrough the Internet or other networks, e.g., the network 280.

As is discussed above, the systems and methods of the present disclosuremay be directed to displaying imaging data that is steganographicallyencoded with one or more visual identifiers on a display device,identifying supplemental information or content relating to the imagingdata based on the one or more visual identifiers, and displaying some orall of the supplemental information or content on another displaydevice, e.g., a display device provided on a mobile device such as asmartphone or tablet computer. Referring to FIG. 3, a flow chart 300 ofone process for steganographic camera communication in accordance withembodiments of the present disclosure is shown. At box 310, using amobile device, a second image is captured of a first image that includesa scene and one or more objects. For example, referring again to FIG.1D, a user, such as the user 130, may orient a smartphone, a tabletcomputer or another mobile device having a camera or other imagingdevice and a display screen toward a video display, a billboard oranother machine or apparatus having digital video content or otherinformation rendered thereon, and may capture one or more still ormoving images of the video content.

At box 320, the captured second image is searched for one or moresteganographically encoded visual identifiers. In accordance with thepresent disclosure, images or imaging data may be searched or otherwiseevaluated in any manner. For example, in some embodiments, each andevery image or frame may be individually searched for steganographicallyencoded content, such as one or more steganographically encoded visualidentifiers. In some other embodiments, images or frames may be searchedat any interval, e.g., after a predetermined number of images or frameshave passed, or after a predetermined length of time has lapsed.Moreover, the images or frames may be searched in their entirety, or inpredetermined locations (e.g., about perimeters or other areas of therespective images or frames), for one or more steganographically encodedvisual identifiers.

Alternatively, in some embodiments, the mobile device may be configuredto search predetermined locations within or aspects of the images orframes for a steganographically encoded indicator, e.g., a specific orpredefined symbol, character or other type of marking indicative ofwhether an image or frame includes one or more visual identifiers. Insome embodiments, a steganographically encoded indicator may indicatethat a particular image or frame includes one or more steganographicallyencoded visual identifiers elsewhere throughout the image or frame. If apredetermined location of an image or frame does not include theindicator, then it may be assumed that the image or frame does notinclude any visual identifiers, and the entire image or frame need notbe searched, e.g., according to a predetermined strategy or on any otherbasis. If the predetermined location of the image or frame includes theindicator, however, then it may be assumed that the image or frameincludes one or more visual identifiers therein, and the balance of theimage or frame may be searched for such identifiers accordingly.Additionally, such indicators may not only indicate that an image orframe includes one or more visual identifiers steganographically encodedtherein, but also identify a location within the image or frame wheresuch visual identifiers are encoded. Alternatively, a steganographicallyencoded indicator may indicate that a particular image or frame does notinclude any steganographically encoded visual identifiers elsewherethroughout the image or frame. Thus, if a predetermined location of animage or frame includes the indicator, then it may be assumed that theimage or frame does not include any visual identifiers, and the entireimage or frame need not be searched. If the predetermined location ofthe image or frame does not include the indicator, however, then it maybe assumed that the image or frame includes one or more visualidentifiers therein, and the balance of the image or frame may besearched for such identifiers accordingly.

At box 330, whether a visual identifier is found within the capturedsecond image is determined. If a visual identifier is not found withinthe captured second image, then the process ends. If a visual identifieris found, however then the process advances to box 340, where the visualidentifier is interpreted, and to box 350, where an external resourcelinked to the visual identifier is identified. For example, where thevisual identifier is a bar code or a URL, the bar code or URL may bedecoded and recognized as a reference to a network page or otherexternal resource relating to the visual identifier having information,data or other content included therein.

At box 360, information maintained at an external resource is accessedvia the visual identifier. For example, the information may be hosted ata network site or other external resource linked to the visualidentifier, e.g., a bar code or a URL, and may be accessed after thevisual identifier is interpreted. Alternatively, in some embodiments,the visual identifier may be a set of text or other charactersrepresenting supplemental information or content relating to the imagingdata, and need not be maintained at any external resource. At box 370,the information is displayed in association with the scene or one ormore objects in the second image on a display of the mobile device, andthe process ends. For example, the information may be rendered in awindow or other feature superimposed upon the second image on thedisplay, such as is shown in FIG. 1D. Alternatively, where the visualidentifier is a bar code or a URL associated with a network page, thenetwork page may replace the second image on the display of mobiledevice, e.g., by loading and rendering the network page thereon.

As is discussed above, visual identifiers may be steganographicallyencoded into imaging data by generating patterns of contrast betweenadjacent portions of the imaging data that are visible only to imagingdevices and associated computer processors, and not to humans. Bysteganographically altering one or more least significant bits or lesssignificant bits of representative pixels within a region of an image ina pattern that is consistent with a bar code, a symbol, one or morealphanumeric characters, or other visual identifiers or portionsthereof, information regarding content of the image, or links to suchinformation, may be made available to users of imaging devices that areconfigured to capture images thereof and scan and/or interpret suchimages accordingly.

Referring to FIGS. 4A through 4C, views of aspects of one system orprocess for steganographic camera communication in accordance withembodiments of the present disclosure are shown. Except where otherwisenoted, reference numerals preceded by the number “4” shown in FIGS. 4Athrough 4C indicate components or features that are similar tocomponents or features having reference numerals preceded by the number“2” shown in FIG. 2, or by the number “1” shown in FIGS. 1A through 1D.

As is shown in FIG. 4A, a visual identifier 42 (viz., a Data Matrixtwo-dimensional bar code) that is linked to or associated with contentof a visual image 442 may be steganographically encoded into a portion443 of the visual image 442. The portion 443 of the visual image 442 maybe selected on any basis, including a relation to the content of thevisual image 442, any characteristics or qualities of the representativepixels in the portion 443, or any other relevant factor. Amounts orextents to which such pixels must be modified may be determined based onthe attributes of the visual identifier 42. For example, as is shown inFIG. 4B, the visual identifier 42 comprises an eighteen-by-eighteen gridof dark or light sections or segments. An eighteen-by-eighteen cellarray 45 of corresponding sections or segments including amounts orextents to which pixels within such sections or segments must besteganographically modified in order to encode the visual identifier 42into the visual image 442 may be determined on any basis, including butnot limited to the characteristics or qualities of the representativepixels to be modified, as well as minimum or maximum thresholds fordetectability or invisibility, or any other factor.

For example, as is shown in FIG. 4B, the array 45 indicates that pixelvalues in sections or segments corresponding to darkened (or black)areas of the visual identifier 42 shall be increased by a value of +n(e.g., in binary or decimal, according to a hexadecimal model or anyother model), while pixel values in sections or segments correspondingto lightened (or white) areas of the visual identifier 42 shall bedecreased by a value of −n. Those of ordinary skill in the pertinentarts will recognize that contrasts may be steganographically encodedinto a visual image in any manner and by any extent. For example,referring again to the array 45 of FIG. 4B, alternatively, pixel valuesin sections or segments corresponding to lightened (or white) areas ofthe visual identifier 42 may be increased, and pixel values in sectionsor segments corresponding to darkened (or black) areas of the visualidentifier 42 may be decreased, in order to generate the same contrast.Moreover, the extent to which a section or segment corresponding todarkened (or black) areas of the visual identifier 42 is modified neednot be equal in magnitude and opposite in direction to the extent towhich a section or segment corresponding to lightened (or white) areasof the visual identifier 42 is modified. Furthermore, where the portion443 of the visual image 442 into which the visual identifier 42 is to beencoded varies in brightness, saturation or hue, representative pixelswithin the portion 443 may be modified by various amounts or to variousextents, as necessary, in order to encode the visual identifier 42therein.

As is shown in FIG. 4C, once the amounts or extents to which therepresentative pixels within the portion 443 are to be modified isdetermined, e.g., as is expressed in the array 45, the least significantbits or less significant bits of such pixels may be steganographicallymodified in a manner consistent with the array 45, in order to form amodified visual image 444 with the visual identifier 42 virtuallyencoded therein. Subsequently, an imaging device that is configured tocapture imaging data from the modified visual image 444 may recognizeand interpret the visual identifier 42, and identify supplementalinformation relating to the visual image 442.

As is also discussed above, visual identifiers such as bar codes,alphanumeric characters formed into text or hyperlinks, or any othercontent, may be encoded into any portion of visual imaging data,including but not limited to locations within the visual imaging datathat pertain or relate to specific aspects or portions of the visualimaging data, such as one or more objects depicted within such aspectsor portions. Referring to FIGS. 5A through 5D, views of aspects of onesystem or process for steganographic camera communication in accordancewith embodiments of the present disclosure are shown. Except whereotherwise noted, reference numerals preceded by the number “5” shown inFIGS. 5A through 5D indicate components or features that are similar tocomponents or features having reference numerals preceded by the number“4” shown in FIGS. 4A through 4C, by the number “2” shown in FIG. 2, orby the number “1” shown in FIGS. 1A through 1D.

As is shown in FIG. 5A, visual imaging data 542 includes a plurality ofcommercial products 50A, 50B, 50C, 50D, 50E, including wall paint, asafety helmet, an interior door, a child's outfit and floor tile. Theplurality of commercial products 50A, 50B, 50C, 50D, 50E are representedwithin the visual imaging data 542 by a plurality of colors. Forexample, the wall paint 50A is in an akaroa color having red, green andblue color values of (220, 194, 168) in decimal, or (11011100, 11000010,10101000) in binary; the safety helmet 50B has portions in a royal bluecolor having red, green and blue color values of (77, 77, 222) indecimal, or (01001101, 01001101, 11011110) in binary; the interior door50C is a gallery white color having red, green and blue color values of(243, 243, 243) in decimal, or (11110011, 11110011, 11110011) in binary;the child's outfit 50D has portions in a national flag blue color havingred, green and blue color values of (49, 49, 74) in decimal, or(00110001, 00110001, 01001010) in binary; and the floor tile 50E hasportions in a tana color having red, green and blue color values of(219, 219, 195) in decimal, or (11011011, 11011011, 11000011) in binary.

In some embodiments, visual identifiers that include or are linked to orotherwise associated with supplemental information or contentcorresponding to one or more aspects or portions of visual imaging datamay be embedded into the visual imaging data, e.g., into one or morestill images or moving images. For example, as is shown in FIG. 5B, aplurality of visual identifiers 52A, 52B, 52C, 52D, 52E corresponding toeach of the commercial products 50A, 50B, 50C, 50D, 50E are shown. Thevisual identifiers 52A, 52B, 52C, 52D, 52E shown in FIG. 5B are barcodes that have been encoded in association with supplementalinformation or content, e.g., links to one or more external resourcessuch as network pages or other sources of data regarding the commercialproducts 50A, 50B, 50C, 50D, 50E. For example, the visual identifier 52Ais a two-dimensional Data Matrix bar code, while the visual identifier52B is a one-dimensional PDF 417 bar code. The visual identifier 52C isa one-dimensional Code 39 bar code, while the visual identifier 52D is atwo-dimensional MaxiCode bar code and the visual identifier 52E is aone-dimensional Code 128 bar code. Each of the visual identifiers 52A,52B, 52C, 52D, 52E may be encoded with supplemental information orcontent, or with links to supplemental information or content, and maythemselves be encoded into portions of the visual imaging data 542corresponding to the respective portions of the respective commercialproducts 50A, 50B, 50C, 50D, 50E, and may thereby form modified visualimaging data 544 that may be stored in one or more data stores.

For example, as is shown in FIG. 5C, the visual identifier 52A may beencoded into a portion of the visual imaging data 542 includingrepresentative pixels corresponding to the paint 50A. In someembodiments, as is discussed above with regard to FIGS. 4A through 4C,one or more least significant bits or less significant bits of each ofthe red, green and blue color channels of selected representative pixelsin a portion 543A of the visual imaging data 542 into which the visualidentifier 52A is to be encoded may be modified slightly, as necessary,in order to generate contrasts between and within such pixels forrendering the visual identifier 52A. A pattern corresponding to thevisual identifier 52A may be encoded into the portion 543A of the visualimaging data 542 by changing values of the least significant bits orless significant bits of such pixels to zero or one, as necessary, whichmay cause the visual identifier 52A to be visible to one or more imagingdevices and/or computer-based processing applications or components, yetremain invisible to the human eye. For example, as is shown in FIG. 5C,least significant bits or less significant bits of one or more of theakaroa pixels of the portion 543A of the visual imaging data 542 may bemodified to cause the visual identifier 52A to be encoded therein, e.g.,by converting such pixels to one or more brown colors having red, greenand blue color values with ranges between (216, 192, 168) and (223, 199,175) in decimal, or between (11011000, 11000000, 10101000) and(11011111, 11000111, 10101111) in binary. Likewise, least significantbits of one or more of the royal blue, gallery white, national flag blueor tana pixels of the visual imaging data may be modified to cause thevisual identifiers 52B, 52C, 52D, 52E to be encoded therein, e.g., byconverting pixels in a portion 543B to one or more blue colors havingred, green and blue color values with ranges between (72, 72, 216) and(79, 79, 223) in decimal, pixels in a portion 543C to one or more whitecolors having red, green and blue color values with ranges between (240,240, 240) and (247, 247, 247) in decimal, pixels in a portion 543D toone or more blue colors having red, green and blue color values withranges between (48, 48, 72) and (55, 55, 79) in decimal, and pixels in aportion 543E to one or more tan colors having red, green and blue colorvalues with ranges between (216, 216, 192) and (223, 223, 199) indecimal, respectively.

Once the modified imaging data 544 has been formed by encoding thevisual identifiers 52A, 52B, 52C, 52D, 52E into the portions 543A, 543B,543C, 543D, 543E of the visual imaging data 542, the modified imagingdata 544 may be scanned, evaluated or photographed using one or moresmartphones, tablet computers, mobile devices or other computersequipped or associated with one or more imaging devices, and thesupplemental information or content associated with such identifiers52A, 52B, 52C, 52D, 52E may be rendered thereby. For example, as isshown in FIG. 5D, when an image is captured of the imaging data 544, andthe visual identifiers 52A, 52B, 52C, 52D, 52E are recognized andinterpreted, supplemental information or content may be displayed in oneor more windows 54A, 54B, 54C, 54D, 54E. In some embodiments, such as isshown in FIG. 5D, the supplemental information or content displayed uponrecognizing and interpreting one or more visual identifiers may includealphanumeric characters (e.g., text, numbers and/or symbols) embodied insuch identifiers, or text that is maintained in an external resource towhich the identifiers may be linked (e.g., one or more network sites).

Moreover, the supplemental information or content steganographicallyembedded into visual imaging data, or linked to one or more identifiersembedded therein, may be provided for a specific purpose. For example,in the embodiment of FIGS. 5A through 5E, where the visual imaging data542 is a portion of a motion picture (e.g., a digital image or a seriesof digital images), the visual imaging data 542 may be modified byembedding the visual imaging data 542 with identifying informationregarding the one or more commercial products 50A, 50B, 50C, 50D, 50Eshown in the visual imaging data 542, or information regarding pricingor availability of the commercial products 50A, 50B, 50C, 50D, 50E.Thus, when a viewer of the modified imaging data 544 shown in FIG. 5Dcaptures an image of the modified imaging data 544 using a mobile deviceequipped with a camera or other imaging device, such as the smartphone132 shown in FIG. 1D, the visual identifiers 52A, 52B, 52C, 52D, 52E maybe recognized and interpreted, and the supplemental information orcontent associated with such identifiers 52A, 52B, 52C, 52D, 52E may beretrieved and shown in the windows 54A, 54B, 54C, 54D, 54E. In someother embodiments, the windows 54A, 54B, 54C, 54D, 54E may contain linksto one or more network sites or other external resources having featureswhich enable a customer to purchase one or more of the respectivecommercial products 50A, 50B, 50C, 50D, 50E. In still other embodiments,the supplemental information or content may be of any type or form, andneed not relate to any commercial function or purpose.

As is discussed above, a visual identifier including alphanumericcharacters, symbols, optically readable identifiers (e.g., bar codes) orany other content may be steganographically encoded into one or morediscrete locations within a set of visual data, e.g., by selecting suchlocations, identifying a predetermined number of least significant bitsor less significant bits within representative pixels in such locations,and modifying such bits to accommodate the visual identifier in a mannerthat creates a visual contrast that may be recognized by an imagingdevice and one or more associated computer components or modules, butremains invisible to the human eye. Referring to FIG. 6, a flow chart600 of one process for steganographic camera communication in accordancewith embodiments of the present disclosure is shown. At box 610, imagingdata having a scene with one or more objects therein is identified. Theimaging data may be captured and stored in one or data stores, indexed,and subsequently retrieved, or may be captured and identified asincluding the scene and the one or more objects therein in real time orin near-real time. In some embodiments, the imaging data may be a singlevisual image. In other embodiments, the imaging data may include one ormore frames of visual imaging data, e.g., motion pictures.

At box 620, supplemental content associated with the scene and/or theobjects that is to be encoded into the imaging data is determined. Forexample, the supplemental content may include descriptors of the scenein terms of its real or fictional location, as well as any relevanttimes, dates or prevailing weather conditions (e.g., temperatures,humidities or precipitation) at the scene. The supplemental content mayalso include descriptors of the objects, e.g., names, categories orother labels of such objects, which may be structures, vehicles,machines, animals (e.g., humans) or any other type of object that may bevisibly depicted in imaging data, in whole or in part. At box 630, oneor more links to the supplemental content is generated for each of thescene and/or the objects. For example, referring again to FIGS. 5B and5C, the visual identifiers 52A, 52B, 52C, 52D, 52E may each be generatedand associated with a respective aspect of the imaging data.Alternatively, a single visual identifier may be generated and providedfor the scene and objects in its entirety. In some embodiments, thelinks may be bar codes or hyperlinks to the supplemental content, or toone or more external resources having the supplemental content storedthereon, e.g. network pages and/or one or more remote servers.Alternatively, the supplemental content itself may be steganographicallyencoded into the imaging data.

At box 640, a minimum level of color resolution for the imaging data isdetermined. As is discussed above, the steganographic encoding of datainto imaging data is accomplished at a cost in terms of clarity andresolution. Therefore, a minimum level of color resolution for imagingdata that may be permitted in order to ensure that data is encodedtherein in a manner that is invisible to the human eye may be selected.At box 650, a number and location of bits of data within the imagingdata for encoding links to the supplemental content is selected based onthe minimum level of color resolution and the size of the links. Forexample, as is discussed above, a visual identifier of any type, shapeor form may be encoded into any portion of imaging data. Therefore, aportion of the visual imaging data that is large enough to accommodatethe visual identifier, and includes representative pixels that may beadequately modified in order to generate a minimum level of visualcontrast needed in order for the visual identifier to be recognized andinterpreted, while maintaining the minimum level of resolution, may bechosen accordingly.

At box 660, bits of data within the imaging data are modified tosteganographically encode the links to the supplemental content therein.The links may be encoded in a manner that defines a sufficient level ofvisual contrast, e.g., by co-opting one or more least significant bitsor less significant bits within some of the representative pixelsidentified at box 650. For example, as is discussed above with regard toFIGS. 1A through 1D, or FIGS. 4A through 4C, bits of the red, green andblue color channels of specific pixels in shapes and formats of thevisual identifiers may be modified in order to encode the visualidentifiers therein. At box 670, the modified imaging data is stored inat least one data store, and the process ends.

As is discussed above, visual identifiers may be steganographicallyencoded into any portion of visual imaging data, including into one ormore locations that may be identified or selected on any basis.Referring to FIGS. 7A through 7D, views of aspects of one system orprocess for steganographic camera communication in accordance withembodiments of the present disclosure are shown. Except where otherwisenoted, reference numerals preceded by the number “7” shown in FIGS. 7Athrough 7D indicate components or features that are similar tocomponents or features having reference numerals preceded by the number“5” shown in FIGS. 5A through 5D, by the number “4” shown in FIGS. 4Athrough 4C, by the number “2” shown in FIG. 2, or by the number “1”shown in FIGS. 1A through 1D.

In some embodiments, a visual identifier associated with supplementalcontent may be steganographically encoded into an entire image or frameof imaging data, e.g., where the visual identifier relates to the imageor frame of imaging data as a whole. As is shown in FIG. 7A, a singlelink 72A (e.g., a one-dimensional bar code) may be generally encodedinto a central region 743A of an image frame 742, in order to form amodified image frame 744A. Thus, when the image frame 742 is displayedon a computer display, or printed into hard copy, and one or more imagesof the modified image frame 744A is captured, e.g., using a smartphone,a tablet computer or other computer device equipped or associated withan imaging device, the link 72A may be recognized and interpreted inorder to access supplemental content regarding the image frame 742and/or the contents thereof. For example, as is discussed above,representative pixels consistent with a pattern of the link 72A may beselectively altered within the region 743A in order to create visualcontrasts that may be interpreted by imaging devices and/or computerdevices, but are imperceptible to the human eye.

In some other embodiments, visual identifiers associated withsupplemental content may be steganographically encoded in remote oreccentric locations within an image or frame of imaging data, e.g.,about a perimeter of the image or frame. Thus, to the extent that thesteganographic encoding impacts a level of clarity or resolution of theimage or frame, any adverse effects of the steganographic encoding maybe disposed in areas of limited visibility within the image or frame. Asis shown in FIG. 7B, a plurality of links 72B-1, 72B-2, 72B-3 (e.g.,one-dimensional and/or two-dimensional bar codes) may be generallyencoded along a right side 743B-1 of the image frame 742, about aperimeter 743B-2 of the image frame 742, viz., along a bottom edge, orin a top left corner 743B-3 of the image frame 742. Representativepixels consistent with patterns of the links 72B-1, 72B-2, 72B-3 may beselectively incorporated into the portions 743B-1, 743B-2, 743B-3 of theimage frame 742 such that the links 72B-1, 72B-2, 72B-3 may beinterpreted by imaging devices and/or computer devices, yet remain inouter portions of the image frame 742, in order to form a modified imageframe 744B.

Likewise, in other embodiments, visual identifiers associated withsupplemental content may be steganographically encoded into regions ofan image or frame of imaging data in which a signal-to-noise ratio isrelatively low. For example, representative pixels or regions of animage for steganographically encoding visual identifiers may be selectedwithin substantially “noisy” portions of the image, e.g., portions ofthe image having irregular or unnatural variations in pixelcharacteristics such as brightness, saturation or hue, as determined byresort to metrics regarding levels of variance or signal-to-noiseratios. Thus, as is discussed above with regard to FIG. 7B, to theextent that the steganographic encoding impacts a level of clarity orresolution of the image or frame, any adverse effects of thesteganographic encoding may be disposed in regions of high or varyingcontrasts, where such adverse effects may not be readily recognized byhumans. As is shown in FIG. 7C, where the image frame 742 includes ascene having alternating or rapidly varying levels of contrast amongflora within the scene, or in clothing worn by persons within the scene,a plurality of links 72C-1, 72C-2, 72C-3 (e.g., one-dimensional and/ortwo-dimensional bar codes) to information regarding the scene and/orobjects therein (e.g., the flora, the persons or the clothing) may begenerally encoded in regions 743C-1, 743C-2, 743C-3 having the lowsignal-to-noise ratios and/or varying levels of contrast in order toform a modified image frame 744C.

In still other embodiments, such as the embodiments discussed above withregard to FIGS. 5A through 5D, visual identifiers associated withsupplemental content may be steganographically encoded into regions ofimaging data to which the supplemental content is particularly relevant.As is shown in FIG. 7D, where the image frame 742 includes a scenehaving various commercial products or other objects such as vegetationor specific articles of clothing, a plurality of links 72D-1, 72D-2,72D-3, 72D-4, 72D-5, 72D-6 (e.g., one-dimensional and/or two-dimensionalbar codes) may be generally encoded in regions 743D-1A, 743D-1B, 743D-2,743D-3, 743D-4, 743D-5, 743D-6 of the image frame 742 corresponding tothe products or objects, in order to form a modified image frame 744D.

As is also discussed above, visual identifiers that aresteganographically encoded into visual imaging data may includealphanumeric characters such as text, numbers or symbols. Suchcharacters may represent links to supplemental content regarding thevisual imaging data or the contents thereof, or may include supplementalcontent itself. Referring to FIGS. 8A and 8B, views of aspects of onesystem or process for steganographic camera communication in accordancewith embodiments of the present disclosure are shown. Except whereotherwise noted, reference numerals preceded by the number “8” shown inFIGS. 8A and 8B indicate components or features that are similar tocomponents or features having reference numerals preceded by the number“7” shown in FIGS. 7A through 7D, by the number “5” shown in FIGS. 5Athrough 5D, by the number “4” shown in FIGS. 4A through 4C, by thenumber “2” shown in FIG. 2, or by the number “1” shown in FIGS. 1Athrough 1D.

As is shown in FIG. 8A, a visual identifier 82A in the form of a URL orother text-based reference to an external resource (e.g., one or morenetwork pages) may be steganographically encoded into a portion 843A ofan image frame 842, such as in or about a perimeter of the image frame842, in order to form a modified image frame 844A. Thus, when themodified image frame 844A is displayed on a computer display, or printedinto hard copy, and one or more images of the modified image frame 844Aare captured, e.g., using a smartphone, a tablet computer or othercomputer device equipped or associated with an imaging device, the link82A may be recognized and interpreted, and supplemental contentregarding the image frame 842 and/or the contents thereof may beaccessed accordingly.

As is shown in FIG. 8B, visual identifiers 82B-1, 82B-2, 82B-3, 82B-4,82B-5, 82B-6 in the form of text-based labels or descriptors of contentmay be steganographically encoded into portions 843B-1, 843B-2, 843B-3,843B-4, 843B-5A, 843B-5B, 843B-6 of the image frame 842, in order toform a modified image frame 844B. For example, where the image frame 842depicts a child, a statue and various plant life, the visual identifier82B-3 corresponding to a label or descriptor of the child, the visualidentifier 82B-2 corresponding to a label or descriptor of the statue,or the visual identifiers 82B-1, 82B-4, 82B-5 corresponding to labels ordescriptors of the plant life, may be encoded into the image frame 842.Thus, when the modified image frame 844B is displayed on a computerdisplay, or printed into hard copy, and one or more images of themodified image frame 844B is captured, e.g., using a smartphone, atablet computer or other computer device equipped or associated with animaging device, the visual identifiers 82B-1, 82B-2, 82B-3, 82B-4,82B-5, 82B-6 may be recognized and interpreted, and supplemental contentregarding the image frame 842 and/or the contents thereof may berecognized and displayed accordingly.

As is also discussed above, visual identifiers or other references tosupplemental information or content may be steganographically encodedinto video files, e.g., into one or more frames of moving images. Thus,as the video files are shown by a display system, a viewer of the videofiles may capture one or more images of the video files, and thesteganographically encoded visual identifiers may be recognized andinterpreted in order to access the supplemental information or contentregarding the video files. Referring to FIG. 9, views of aspects of onesystem or process for steganographic camera communication in accordancewith embodiments of the present disclosure are shown. Except whereotherwise noted, reference numerals preceded by the number “9” shown inFIG. 9 indicate components or features that are similar to components orfeatures having reference numerals preceded by the number “8” shown inFIGS. 8A and 8B, by the number “7” shown in FIGS. 7A through 7D, by thenumber “5” shown in FIGS. 5A through 5D, by the number “4” shown inFIGS. 4A through 4C, by the number “2” shown in FIG. 2, or by the number“1” shown in FIGS. 1A through 1D.

As is shown in FIG. 9, a plurality of image frames 942A, 942B, 942C,942D, 942E depicting a child playing basketball on an indoor hoop areshown in series, e.g., as portions of a video file. As is also shown inFIG. 9, the indoor hoop on which the child is playing is offered forsale via a network page 916 from which customers may execute a purchaseof the indoor hoop. Thus, a shortened URL 92 associated with the networkpage 916 may be embedded into one or more of the image frames 942A,942B, 942C, 942D, 942E in series. For example, the image frames 942A,942B, 942C, 942D, 942E of FIG. 9 are portions of a ten-second clip 944,and the shortened URL 92 is embedded into a six-second segment of theten-second clip 944 that begins two seconds after a start of theten-second clip 944 and ends two seconds prior to an end of theten-second clip 944. Therefore, as the ten-second clip 944 is shown on adisplay, a user of a computer device equipped or associated with animaging device who captures an image of the six-second segment of theten-second clip 944 may cause the network page 916 or portions thereofto be rendered on a display via the shortened URL 92 that issteganographically embedded within the segment.

As is discussed above, visual identifiers that are steganographicallyencoded into imaging data may be varied both spatially and temporally,such that visual identifiers may be encoded and viewed in differentlocations with one or more rendered or displayed images, or rendered ordisplayed at different times within one or more of such images.Referring to FIG. 10, views of aspects of one system or process forsteganographic camera communication in accordance with embodiments ofthe present disclosure are shown. Except where otherwise noted,reference numerals preceded by the number “10” shown in FIG. 10 indicatecomponents or features that are similar to components or features havingreference numerals preceded by the number “9” shown in FIG. 9, by thenumber “8” shown in FIGS. 8A and 8B, by the number “7” shown in FIGS. 7Athrough 7D, by the number “5” shown in FIGS. 5A through 5D, by thenumber “4” shown in FIGS. 4A through 4C, by the number “2” shown in FIG.2, or by the number “1” shown in FIGS. 1A through 1D.

As is shown in FIG. 10, a plurality of image frames 1042A, 1042B, 1042C,1042D, 1042E, 1042F depicting a child dribbling a basketball outdoorsare shown in series, e.g., as portions of a video file. As is also shownin FIG. 10, a plurality of additional information 1002A, 1002B, 1002C(viz., the words “BOUNCING RUBBER BALL”) associated with the content ofthe image frames 1042A, 1042B, 1042C, 1042D, 1042E, 1042F is identified,and embedded into the image frames 1042A, 1042B, 1042C, 1042D, 1042E,1042F to form a modified set of imaging data 1044. In accordance withthe present disclosure, the additional information 1002A, 1002B, 1002Cmay be encoded into different spatial locations within such frames, inorder to ensure that no single region of representative pixels withinthe modified set of imaging data 1044 is persistently modified to encodethe additional information therein. For example, as is shown in FIG. 10,the image frames 1042A, 1042B are modified to steganographically encodethe additional information 1002A (viz., the word “BOUNCING”) into anupper left corner of the image frames 1042A, 1042B. The image frames1042C, 1042D are also modified to steganographically encode theadditional information 1002B (viz., the word “RUBBER”) into a centralportion of the image frames 1042C, 1042D. The image frames 1042E, 1042Fare further modified to steganographically encode the additionalinformation 1002C (viz., the word “BALL”) into a lower right portion ofthe image frames 1042E, 1042F. Thus, by varying the spatial locations ofrepresentative pixels that are modified to accommodate additionalinformation (e.g., one or more bar codes, alphanumeric characters,symbols or other features), the limited risk that the additionalinformation may be recognized by a fovea of a human eye may bemitigated.

Likewise, the risk may also be further mitigated by displaying frameshaving steganographically encoded identifiers at different times.Referring to FIG. 11, views of aspects of one system or process forsteganographic camera communication in accordance with embodiments ofthe present disclosure are shown. Except where otherwise noted,reference numerals preceded by the number “11” shown in FIG. 11 indicatecomponents or features that are similar to components or features havingreference numerals preceded by the number “10” shown in FIG. 10, by thenumber “9” shown in FIG. 9, by the number “8” shown in FIGS. 8A and 8B,by the number “7” shown in FIGS. 7A through 7D, by the number “5” shownin FIGS. 5A through 5D, by the number “4” shown in FIGS. 4A through 4C,by the number “2” shown in FIG. 2, or by the number “1” shown in FIGS.1A through 1D.

As is shown in FIG. 11, a plurality of image frames 1142A, 1142B, 1142C,1142D, 1142E depicting a child playing with a toy rocket are shown inseries, e.g., as portions of a video file. As is also shown in FIG. 11,a plurality of visual identifiers 1102A, 1102B, 1102C associated withthe content of the image frames 1142A, 1142B, 1142C, 1142D, 1142E isidentified, and embedded into the image frames 1142A, 1142B, 1142C,1142D, 1142E to form a modified set of imaging data 1144. Theidentifiers 1102A, 1102B, 1102C may relate to the child, the toy rocket,the scene or any other element expressed in the image frames 1142A,1142B, 1142C, 1142D, 1142E. In accordance with the present disclosure,the visual identifiers 1102A, 1102B, 1102C may be encoded into commonportions of different frames, in order to ensure that no single regionof representative pixels within the modified set of imaging data 1144 ispersistently modified to encode the additional information therein. Forexample, as is shown in FIG. 11, the image frame 1142A is modified tosteganographically encode the visual identifier 1102A into a centralportion of the image frame 1142A. The image frame 1142C is also modifiedto steganographically encode the visual identifier 1102B into thecentral portion of the image frame 1142C, and the image frame 1142E isfurther modified to steganographically encode the visual identifier1102C into the same central portion of the image frame 1142E. Thus, byvarying the times at which representative pixels are modified toaccommodate visual identifiers therein, the limited risk that theadditional information may be recognized by a fovea of a human eye maybe further mitigated.

Those of ordinary skill in the pertinent arts will recognize thatimaging data may be steganographically encoded to include visualidentifiers and/or additional information both in different spatiallocations and at different times in accordance with the presentdisclosure, e.g., by combining the processes or techniques described inFIGS. 10 and 11, with respect to any type of visual identifier oradditional information, any spatial location in one or more imageframes, or at any time.

As is discussed above, when an image of imaging data is captured inaccordance with the present disclosure, the image may be searched in itsentirety to determine whether the image includes any steganographicallyencoded visual identifiers therein. Alternatively, one or more discreteor select portions of an image may be searched for visual identifiers,or for one or more telltale indicators that the image includes one ormore visual identifiers. If no such indicators are found, then a balanceof the image need not be searched. If any such indicators are found,however, the balance of the image may be searched, and any visualidentifiers recognized therein may be interpreted in order to identifyand display supplemental information or content associated with thatimage or portions thereof.

Referring to FIG. 12, a flow chart 1200 of one process forsteganographic camera communication in accordance with embodiments ofthe present disclosure is shown. At box 1210, imaging data is capturedusing a mobile device, and searched in order to determine whether theimaging data includes one or more images represented therein. Forexample, the imaging data may be evaluated to determine whether one ormore polygons or other shapes or regions corresponding to a video imageshown on a display are depicted therein, e.g., based on differences inrelative intensities of image pixels within such shapes or regionswithin the imaging data as compared to a remainder of the displayedvideo image in order to determine whether the shapes or regions arevideo images.

At box 1215, whether any images are recognized within one or more framesof the captured imaging data is determined. If no such images arerecognized, then the process advances to box 1280, where it isdetermined whether the continued searching of captured imaging data forimages expressed therein is desired, e.g., in response to a user prompt.If continued searching is not desired, then the process ends. Ifcontinued searching is desired, however, then the process returns to box1210.

If any images are recognized within the one or more frames of thecaptured imaging data, then the process advances to box 1220, where theoutlines of such images are searched for one or more telltale indicatorsof steganographically encoded data. For example, a telltale indicatormay be located in a predetermined portion of the image outline (e.g., aselect corner, a geometric center, or any other location), such that thepresence of the telltale indicator within the predetermined portionindicates that one or more visual identifiers is steganographicallyencoded elsewhere within the image outline, and the absence of thetelltale indicator within the predetermined portion indicates that theimage outline does not include any steganographically encoded visualidentifiers therein. Alternatively, in some embodiments, a telltaleindicator within an image outline may indicate that nosteganographically encoded visual identifiers are encoded therein, whilethe absence of a telltale indicator may indicate that the image outlineincludes one or more visual identifiers. In other embodiments, atelltale indicator may also designate a location of such visualidentifiers within the image outline, e.g., where the telltale indicatorcomprises a symbol or character indicative of locations within theoutline, such as Cartesian coordinates or other labels. At box 1225,whether a telltale indicator is found within the image outline isdetermined. If no telltale indicators are found, then the processadvances to box 1280.

If any telltale indicators are found, however, then the process advancesto box 1230, where the one or more frames of the captured imaging dataare searched for steganographically encoded supplemental content orlinks thereto. For example, after a telltale indicator is found, theimaging data may be searched for visual identifiers such as the bar code12 of FIG. 1C or the bar codes 52A, 52B, 52C, 52D, 52E of FIGS. 5B and5C, the shortened URL 82A of FIG. 8A, or the text-based labels ordescriptors 82B-1, 82B-2, 82B-3, 82B-4, 82B-5, 82B-6 of FIG. 8B. Suchidentifiers may be steganographically encoded into any portion of theimaging data, such as a perimeter, a central location, a corner, anedge, or a region selected based on properties of the imaging data,e.g., an area of low signal-to-noise ratio, or a region of the imagingdata selected at random. At box 1235, whether any supplemental contentor links are found within the frames of the captured imaging data isdetermined. If no content or links are found, then the process advancesto box 1280.

If any supplemental content or links thereto are found, however, thenthe process advances to box 1240, where the supplemental content orlinks are interpreted. For example, where a hyperlink or bar codeassociated with supplemental content residing on external resources isfound within imaging data, the hyperlink or bar code may be interpreted,and the supplemental content residing thereon may be accessedaccordingly. Where the supplemental content itself is found within theimaging data, however, the supplemental content may be interpreted. Atbox 1250, the supplemental content is displayed with the capturedimaging data on the mobile device. The supplemental content may bemodified, formatted, tabulated or prepared for display in any manner,e.g., within one or more windows or other user interfaces, such as thewindows 54A, 54B, 54C, 54D, 54E shown in FIG. 5D, and displayed on themobile device.

At box 1260, whether an instruction to access an external resourceassociated with the supplemental content has been received isdetermined. For example, referring again to FIG. 1D, the window 14 isdisplayed over the imaging data 144 on the smartphone 132 and includesboth identifying information regarding the bicycle 10A shown in theimaging data as well as one or more interactive features (viz., “OK” or“CANCEL” buttons) for accessing further information regarding thebicycle 10A. Selecting the “OK” button may cause the network page 116 orother information regarding the bicycle 10A to be displayed on themobile device 132. If an instruction to access the external resourceassociated with the supplemental content is not received, e.g., eitherin response to a selection of the “CANCEL” button shown in FIG. 1D, orfollowing a time-out after no response is received, then the processadvances to box 1270, where the supplemental content is removed from themobile device, and to box 1280, where it is determined whether thecontinued searching for images within captured imaging data is desired.If the instruction to access the external resource associated with thesupplemental content is received, however, then the process advances tobox 1265, where the external resource is displayed on the mobile device,and the process ends.

One example in which a telltale indicator may be steganographicallyencoded into visual imaging data and used to quickly indicate whetherthe visual imaging data includes one or more steganographically encodedvisual identifiers therein is shown with regard to FIG. 13. Referring toFIG. 13, views of aspects of one system or process for steganographiccamera communication in accordance with embodiments of the presentdisclosure are shown. Except where otherwise noted, reference numeralspreceded by the number “13” shown in FIG. 13 indicate components orfeatures that are similar to components or features having referencenumerals preceded by the number “11” shown in FIG. 11, by the number“10” shown in FIG. 10, by the number “9” shown in FIG. 9, by the number“8” shown in FIGS. 8A and 8B, by the number “7” shown in FIGS. 7Athrough 7D, by the number “5” shown in FIGS. 5A through 5D, by thenumber “4” shown in FIGS. 4A through 4C, by the number “2” shown in FIG.2, or by the number “1” shown in FIGS. 1A through 1D.

As is shown in FIG. 13, visual identifiers 132A, 132B, 132C, 132D, 132Emay be steganographically encoded into a visual image 1342 to form amodified image 1344. The visual identifiers 132A, 132B, 132C, 132D mayconstitute or relate to supplemental content associated with the visualimage 1342 and/or its contents. The visual indicator 132E is a telltaleindicator (viz., a symbol, such as a star) that is alsosteganographically encoded into the image 1342 along with the visualidentifiers 132A, 132B, 132C, 132D. Thus, when an image is captured ofthe modified image 1344 by a computer equipped or configured with one ormore imaging devices, such as a smartphone or tablet computer, thecomputer may be configured to search a portion of the modified image1344, such as the upper left corner, to determine whether that portionof the modified image 1344 includes a telltale indicator, e.g., thevisual identifier 132E. If no such indicator is found, then a remainderor balance of the modified image 1344 other than the upper left cornerneed not be searched. If such an indicator is found, however, then theremainder or the balance of the modified image 1344 will be searched forone or more visual identifiers, e.g., the visual identifiers 132A, 132B,132C, 132D, which may then be recognized, interpreted and/or displayedby the computer. Alternatively, the presence of the visual identifier132E within the modified image 1344 may indicate that no othersteganographically encoded visual identifiers are encoded therein, ormay identify or suggest a location of such visual identifiers within themodified image 1344.

Configuring a computer such as a smartphone, a tablet computer oranother mobile device that is equipped or configured with one or moreimaging devices to search first for telltale indicators in predeterminedlocations of image files may greatly enhance the speed and efficiencywith which supplemental content is displayed in association with suchimage files. In some embodiments, searching predetermined locations fortelltale indicators may be particularly useful for evaluating videofiles, e.g., a plurality of moving images. If a telltale indicator isnot found within the predetermined locations, then the remainder of theimages need not be searched, thereby freeing up processing power,available memory and/or bandwidth for other purposes. If one or moretelltale indicators are found, however, then the remainder of the imagesshould be searched in order to locate one or more visual identifierstherein. Although the visual identifier 13E of FIG. 13 is shown in theform of a five-pointed star in the upper left corner of the modifiedimage 1344, those of ordinary skill in the pertinent arts will recognizethat a telltale indicator may be any other shape, or may include anynumber of characters (e.g., text, numbers or symbols), or may be locatedin any location within imaging data, in accordance with embodiments ofthe present disclosure, and that such embodiments are not limited tostar-shaped indicators provided in upper left portions of image outlinesor frames.

In some embodiments of the present disclosure, visual indicators ofsupplemental information or content may be steganographically encodedinto live video feeds, e.g., in real time or in near-real time, prior tobroadcasting the live video feeds to viewers. For example, because manytelevised events are aired in accordance with a mandatory broadcastdelay of a predetermined duration, e.g., six to seven seconds, in orderto scrub or block any profanity or objectionable content that may havebeen captured prior to broadcast, the video feeds may be modified toinclude any steganographic visual identifiers associated with thecontent therein during the period associated with the broadcast delay.

Referring to FIG. 14, a flow chart 1400 of one process forsteganographic camera communication in accordance with embodiments ofthe present disclosure is shown. At box 1410, live feeds of imaging dataare captured from an event using one or more imaging devices. Forexample, the event may an evening newscast, a sporting event, a musicalperformance, a press conference, or any other event in which theactivities at the event are covered by one or more imaging devices(e.g., cameras) that are aligned to capture imaging data during theevent and distribute the imaging data for broadcast in near-real time.

At box 1420, a predefined broadcast delay period may be determined. Forexample, the delay period may be a minimum requirement established bylaw or regulation. Alternatively, the broadcast delay may be establishedas a matter of internal policy, or governed by physical or virtualrestrictions on the preparation, transmission, capture and/or display ofsuch visual data. At box 1430, the content of one or more of the livefeeds is evaluated, e.g., using one or more computer processors.

At box 1440, whether the content of the one or more live feeds includesany profanity or other objectionable content is determined. If anyprofanity or other objectionable content is detected within the contentof the one or more live feeds, then the process advances to box 1445,where the content of the live feeds is scrubbed, and to box 1495, whereit is determined whether the event has concluded. If the event has notconcluded, then the process returns to box 1410, where the live feedscontinue to be captured. If the event has concluded, however, then theprocess ends.

If no profanity or other objectionable content has been identified inthe content of the live feeds, then the process advances to box 1450,where supplemental content associated with the content of the live feedsis identified. Such supplemental content may be information or data thatpertains to the live feeds in any way, e.g., to a scene or setting fromwhich the imaging data in the live feeds is captured, or to one or moreof the objects depicted or represented therein. At box 1460, one or morelinks to the supplemental content for the content of the live feeds aregenerated. For example, the links may direct to one or more externalresources where the supplemental content may be found, or may beconfigured to trigger the execution of one or more intrinsic processeswithin an imaging device and/or computer device that captures them.Alternatively, in some embodiments, a visual identifier including thesupplemental content itself may be generated.

At box 1470, a number and location of bits of data within the live feedsfor encoding the link(s) therein are selected. For example, a number ofthe bits of data may be selected based on the size, shape, dimensionsand/or content of the link(s). Likewise, the location of such bits maybe a standard, predetermined location (e.g., a common location withinone or more frames of the imaging data where viewing software may betrained to search), a general location (e.g., centrally located withinframes of the imaging data), or in locations pertaining to the contentto which the linked supplemental content relates, such as is shown inFIGS. 5A and 5C, FIG. 8B or FIG. 13. At box 1480, the bits of data inthe location and number selected at box 1470 within the live feeds aremodified to steganographically encode the links to the supplementalcontent therein. For example, as is discussed above, one or more leastsignificant bits or less significant bits of color channels ofrepresentative pixels in the regions selected at box 1470 may be alteredin order to create a contrast that defines the links, and is visible ormay be interpreted by one or more computer devices, but is imperceptibleto the human eye.

At box 1490, the modified live feeds are broadcast to one or moredisplay devices at the conclusion of the predefined broadcast delay. Forexample, where a broadcast provider has implemented a nominal period ofdelay (e.g., seven seconds) between when imaging data constituting alive feed has been captured, and when the imaging data of the live feedis scheduled to be broadcast, e.g., to enable for a review of theimaging data for profanity or objectionable content, such as at box1440, the imaging data may be modified to include one or more links tosupplemental content within the period of delay and broadcast as “live,”in near-real time, on schedule. Alternatively, a longer period of delaymay be imposed, as necessary, in order to accommodate the identificationof supplemental content, the generation of links to the supplementalcontent, and the selection and modification of bits of data tosteganographically include such links. At box 1495, whether the eventhas concluded is determined. If the event has not concluded, then theprocess returns to box 1410, where the live feeds continue to becaptured. If the event has concluded, however, then the process ends.

Referring to FIGS. 15A and 15B, views of aspects of one system orprocess for steganographic camera communication in accordance withembodiments of the present disclosure are shown. Except where otherwisenoted, reference numerals preceded by the number “15” shown in FIGS. 15Aand 15B indicate components or features that are similar to componentsor features having reference numerals preceded by the number “13” shownin FIG. 13, by the number “11” shown in FIG. 11, by the number “10”shown in FIG. 10, by the number “9” shown in FIG. 9, by the number “8”shown in FIGS. 8A and 8B, by the number “7” shown in FIGS. 7A through7D, by the number “5” shown in FIGS. 5A through 5D, by the number “4”shown in FIGS. 4A through 4C, by the number “2” shown in FIG. 2, or bythe number “1” shown in FIGS. 1A through 1D.

As is shown in FIG. 15A, one or more imaging devices 1520 may capturelive feeds of imaging data 1542 associated with an event (e.g., asporting event). The live feeds 1542 may be processed in order torecognize the content therein, e.g., based on edges, contours, outlines,colors, textures, silhouettes, shapes or other characteristics ofobjects or portions thereof shown in the live feeds. Based on suchcontent, supplemental content 1526A, 1526B, 1526C may be identified. Thesupplemental content may relate to a setting of the event, one or moreparticipants in the event, one or more environmental conditions orattributes of the event, or activities taking place during the event.After the supplemental content 1526A, 1526B, 1526C has been identified,links 152A, 152B, 152C or other visual identifiers relating to thesupplemental content 1526A, 1526B, 1526C (e.g., one-dimensional barcodes, two-dimensional bar codes, or URLs linked to external resourceshaving access to the supplemental content 1526A, 1526B, 1526C) may begenerated and steganographically encoded into the live feeds 1542, inorder to create a modified live feed 1544. The modified live feed 1544may then be broadcast to one or more subscribers and/or display devices,or accessories thereto, that may be configured to display the modifiedlive feed 1544 thereon, such as by a direct broadcast system 1560 or anyother system or method for transmitting audio and/or video content.

As is shown in FIG. 15B, when the modified live feed 1544 is received ata display device (e.g., a television) 1540 via the direct broadcastsystem 1560, a user of a smartphone 1532 or other device having animaging device (e.g., digital camera) and one or more computerprocessors may be may oriented to capture all or a portion of thedisplay device 1540 and the modified live feed 1544 rendered thereonwithin a field of view. Subsequently, one or more of the links 152A,152B, 152C that are steganographically encoded into the modified livefeed 1544 may be recognized and interpreted by the smartphone 1532, andthe supplemental content 1526A, 1526B, 1526C associated with the links152A, 152B, 152C may be accessed and displayed in a plurality of windows154A, 154B, 154C. For example, as is shown in FIG. 15B, the windows154A, 154B may refer or relate to one or more aspects of the eventsdepicted in the modified live feed 1544 or the participants thereof,while the window 154C may include a user interface 1534 for executingone or more commercial opportunities relating to such events, viz., apurchase of an article of clothing worn by a participant in the eventsdepicted in the modified live feed 1544.

Those of ordinary skill in the pertinent arts will recognize that one ormore of the systems and methods for steganographic communicationdisclosed herein may be utilized in any number of fields. In particular,those of ordinary skill will recognize that visual identifiers and/oradditional information (e.g., supplemental content) may besteganographically encoded into imaging data by anyone who owns orcontrols the imaging data, or that such identifiers, information orcontent may be embedded therein at any time. For example, one or more ofthe systems and methods disclosed herein may be incorporated in or usedby commercial-grade or large-scale image processing applications, andalso by applications operated by traditional users of household computerdevices such as laptop, desktop or tablet computers or smartphones. Whena home movie or other video file is captured, the movie or video filemay be augmented with visual identifiers or additional informationreferring or relating to content of the movie or video file in real timeor near-real time, or at a later time.

Moreover, visual identifiers or additional information may besteganographically encoded into, and removed from, imaging data on anynumber of occasions. For example, when a movie is produced for displayin a theater or other public forum, a first set of visual identifiersmay be steganographically encoded therein. When the movie is laterprepared for release via a streaming media service, the first set ofvisual identifiers may be removed therefrom and a second set of visualidentifiers may be steganographically encoded therein. When the movie isfurther prepared for release in a hard storage format (e.g., a DVD, astick drive or another physically portable format), the second set ofvisual identifiers may be removed therefrom and a third set of visualidentifiers may be steganographically encoded therein. Likewise, withregard to television programs, a first set of visual identifiers may besteganographically encoded therein for a live or first-airing of atelevision program, and a second set of visual identifiers may beencoded therein for a rerun or second airing of the television program,and a third set of visual identifiers may be encoded therein for asyndicated release of the television program. As yet another example,when a print advertisement (e.g., a billboard, a magazine article, asubway placard) is prepared for distribution in a number of locationsand includes an image depicting a number of commercial objects therein,copies of the print advertisement may be steganographically encoded withdifferent visual identifiers and/or additional information relating tosuch commercial objects, including but not limited to identifiers ofdifferent nearby brick-and-mortar locations where such commercialobjects may be purchased.

Moreover, in some embodiments, rather than changing the visualidentifiers that are steganographically encoded into imaging data inorder to link to different sets of additional content, the imaging datamay be associated with different sets of additional content by leavingthe visual identifiers in place, and modifying associations of suchidentifiers with external resources on a back end, e.g., by one or moreserver-side applications, such that the visual identifiers may beassociated with different content when the visual identifiers areinterpreted.

Although the disclosure has been described herein using exemplarytechniques, components, and/or processes for implementing the systemsand methods of the present disclosure, it should be understood by thoseskilled in the art that other techniques, components, and/or processesor other combinations and sequences of the techniques, components,and/or processes described herein may be used or performed that achievethe same function(s) and/or result(s) described herein and which areincluded within the scope of the present disclosure.

For example, although some of the embodiments described herein or shownin the accompanying figures refer to the processing of imaging data thatis in color, e.g., according to an RGB color model, the systems andmethods disclosed herein are not so limited, and may be used to processany type of information or data that is provided in color according toany color model, or in black-and-white or grayscale. In someembodiments, the systems and methods disclosed herein may be used inconnection with three-channel, 24-bit visual imaging data havingsupplemental content encoded therein. In other embodiments, however,visual imaging data having any number of channels or bits thereof may beused to encode content therein. Furthermore, those of ordinary skill inthe pertinent arts will recognize that any type of content (e.g.,supplemental content) may also be encoded into any type or form ofimaging data in accordance with the present disclosure, including butnot limited to color imaging data, black-and-white imaging data,grayscale imaging data or depth imaging data.

Additionally, although some of the embodiments disclosed hereinreference the encoding of supplemental content or links thereto intoimaging data that was previously obtained, those of ordinary skill inthe pertinent arts will readily recognize that any content may besteganographically encoded into any imaging data at any time, even ifthe content and the imaging data are obtained simultaneously. Moreover,content may be steganographically encoded into any type or form of imagefile, including but not limited to computer-generated imagery, and thesystems and methods disclosed herein are not limited to the use of stillor moving images that were captured using one or more imaging devices,such as digital cameras.

It should be understood that, unless otherwise explicitly or implicitlyindicated herein, any of the features, characteristics, alternatives ormodifications described regarding a particular embodiment herein mayalso be applied, used, or incorporated with any other embodimentdescribed herein, and that the drawings and detailed description of thepresent disclosure are intended to cover all modifications, equivalentsand alternatives to the various embodiments as defined by the appendedclaims. Moreover, with respect to the one or more methods or processesof the present disclosure described herein, including but not limited tothe flow charts shown in FIG. 3, 6, 12 or 14, orders in which suchmethods or processes are presented are not intended to be construed asany limitation on the claimed inventions, and any number of the methodor process steps or boxes described herein can be combined in any orderand/or in parallel to implement the methods or processes describedherein. Also, the drawings herein are not drawn to scale.

Conditional language, such as, among others, “can,” “could,” “might,” or“may,” unless specifically stated otherwise, or otherwise understoodwithin the context as used, is generally intended to convey in apermissive manner that certain embodiments could include, or have thepotential to include, but do not mandate or require, certain features,elements and/or steps. In a similar manner, terms such as “include,”“including” and “includes” are generally intended to mean “including,but not limited to.” Thus, such conditional language is not generallyintended to imply that features, elements and/or steps are in any wayrequired for one or more embodiments or that one or more embodimentsnecessarily include logic for deciding, with or without user input orprompting, whether these features, elements and/or steps are included orare to be performed in any particular embodiment.

The elements of a method, process, or algorithm described in connectionwith the embodiments disclosed herein can be embodied directly inhardware, in a software module stored in one or more memory devices andexecuted by one or more processors, or in a combination of the two. Asoftware module can reside in RAM, flash memory, ROM, EPROM, EEPROM,registers, a hard disk, a removable disk, a CD-ROM, a DVD-ROM or anyother form of non-transitory computer-readable storage medium, media, orphysical computer storage known in the art. An example storage mediumcan be coupled to the processor such that the processor can readinformation from, and write information to, the storage medium. In thealternative, the storage medium can be integral to the processor. Thestorage medium can be volatile or nonvolatile. The processor and thestorage medium can reside in an ASIC. The ASIC can reside in a userterminal. In the alternative, the processor and the storage medium canreside as discrete components in a user terminal.

Disjunctive language such as the phrase “at least one of X, Y, or Z,” or“at least one of X, Y and Z,” unless specifically stated otherwise, isotherwise understood with the context as used in general to present thatan item, term, etc., may be either X, Y, or Z, or any combinationthereof (e.g., X, Y, and/or Z). Thus, such disjunctive language is notgenerally intended to, and should not, imply that certain embodimentsrequire at least one of X, at least one of Y, or at least one of Z toeach be present.

Unless otherwise explicitly stated, articles such as “a” or “an” shouldgenerally be interpreted to include one or more described items.Accordingly, phrases such as “a device configured to” are intended toinclude one or more recited devices. Such one or more recited devicescan also be collectively configured to carry out the stated recitations.For example, “a processor configured to carry out recitations A, B andC” can include a first processor configured to carry out recitation Aworking in conjunction with a second processor configured to carry outrecitations B and C.

Language of degree used herein, such as the terms “about,”“approximately,” “generally,” “nearly” or “substantially” as usedherein, represent a value, amount, or characteristic close to the statedvalue, amount, or characteristic that still performs a desired functionor achieves a desired result. For example, the terms “about,”“approximately,” “generally,” “nearly” or “substantially” may refer toan amount that is within less than 10% of, within less than 5% of,within less than 1% of, within less than 0.1% of, and within less than0.01% of the stated amount.

Although the invention has been described and illustrated with respectto illustrative embodiments thereof, the foregoing and various otheradditions and omissions may be made therein and thereto withoutdeparting from the spirit and scope of the present disclosure.

What is claimed is:
 1. A computer-implemented method comprising:identifying an image of at least a portion of a scene by at least onecomputer processor; extracting information regarding at least one of anedge, a contour, an outline, a color, a texture, a silhouette or a shapewithin the image of at least the portion of the scene by the at leastone computer processor; determining that the at least one of the edge,the contour, the outline, the color, the texture, the silhouette or theshape corresponds to an object by the at least one computer processor;identifying content relating to at least one of the scene or the objectby the at least one computer processor, wherein the content relating toat least one of the scene or the object is maintained in at least onedata store; generating a link to the content relating to at least one ofthe scene or the object by the at least one computer processor;modifying the image by the at least one computer processor, whereinmodifying the image comprises steganographically encoding the link intoa portion of the image; and storing the modified image in at least onedata store.
 2. The computer-implemented method of claim 1, whereinmodifying the image further comprises: determining a location of the atleast one of the edge, the contour, the outline, the color, the texture,the silhouette or the shape within the image by the at least onecomputer processor, wherein the portion of the image is the location ofthe at least one of the edge, the contour, the outline, the color, thetexture, the silhouette or the shape within the image; and determining apattern associated with the link by the at least one computer processor;and selecting a plurality of representative pixels in the portion of theimage corresponding to the pattern associated with the link by the atleast one computer processor, wherein the link is steganographicallyencoded into the plurality of representative pixels.
 3. Thecomputer-implemented method of claim 2, wherein steganographicallyencoding the link into the portion of the image comprises: identifying aplurality of bits of at least some of the representative pixels in theportion of the image corresponding to the pattern associated with thelink by the at least one computer processor; and steganographicallymodifying the plurality of bits to generate a visual contrast within theportion of the image, wherein the visual contrast is not visible to ahuman eye.
 4. The computer-implemented method of claim 1, whereinidentifying the content relating to at least one of the scene or theobject comprises: determining that the object is a commercial product bythe at least one computer processor based at least in part on the atleast one of the edge, the contour, the outline, the color, the texture,the silhouette or the shape, wherein the content relating to at leastone of the scene or the object is at least one network page offering thecommercial product for sale.
 5. The computer-implemented method of claim1, wherein identifying the image of at least the portion of the scenecomprises: capturing the image of at least the portion of the scene byat least one imaging device, wherein the portion of the scene is withina field of view of the at least one imaging device; and wherein themethod further comprises: causing a display of the modified image on atleast one display device over a network.
 6. A server comprising: atleast one computer processor; and at least one data store having atleast one or more instructions stored thereon, wherein the server, uponexecuting the one or more instructions, is configured to at least:identify a series of image frames maintained in the at least one datastore; extract information regarding at least one of an edge, a contour,an outline, a color, a texture, a silhouette or a shape depicted withinat least the subset of the series of image frames; determine that the atleast one of the edge, the contour, the outline, the color, the texture,the silhouette or the shape depicted within at least the subset of theseries of image frames depicts at least one item in at least a portionof a scene; identify content relating to the at least one item; generateat least one link to at least a portion of the content;steganographically embed the at least one link into portions of each ofthe image frames of the subset depicting the at least one of the edge,the contour, the outline, the color, the texture, the silhouette or theshape; store a video file comprising the series of image frames in theat least one data store, wherein the video file comprises the subset ofthe series of image frames having the at least one linksteganographically embedded therein; transmit, over at least onecomputer network, the video file to a first device comprising at least afirst display; receive, over the at least one computer network, arequest for at least the portion of the content from a second devicecomprising at least one camera, wherein the request comprises the link;and transmit, over the at least one computer network, at least some ofthe content to the second device.
 7. The server of claim 6, wherein theserver, upon executing the one or more instructions, is furtherconfigured to at least: determine a pattern associated with the at leastone link; select a plurality of representative pixels in each of thesubset of the series of image frames corresponding to the pattern;identify a plurality of bits of at least some of the plurality ofrepresentative pixels in each of the subset of the series of imageframes corresponding to the pattern; and modify the plurality of bits ineach of the subset of the series of image frames to generate a visualcontrast that is not visible to the human eye.
 8. The server of claim 6,wherein the link is one of: a uniform resource locator of at least onenetwork site having the content; a bar code having the uniform resourcelocator of the at least one network site encoded therein; or a pluralityof characters comprising at least some of the content.